Original: Therefore, we have constructed a consistent algorithm
Revised: Therefore, we have constructed a consistent algorithm

Original: A for H .
Revised: A for H .

Original: We have completed this proof.
Revised: We have completed this proof.

Original: I.2 Proof of Theorem
Revised: I.2 Proof of Theorem

Original: Theorem 7.
Revised: Theorem 7.

Original: Let |X| < + ∞ and H
Revised: Let |X| < + ∞ and H

Original: = H in • H b .
Revised: = H in • H b .

Original: If H all − { h out } ⊂ H b and Condition 2 holds, then OOD detection is learnable in D sXY for H , where H all and h out are defined in Theorem 6.
Revised: If H all − { h out } ⊂ H b and Condition 2 holds, then OOD detection is learnable in D sXY for H , where H all and h out are defined in Theorem 6.

Original: Proof of Theorem 7.
Revised: Proof of Theorem 7.

Original: Since |X| < + ∞ , we know that |H| < + ∞ , which implies that H in is agnostic PAC learnable for supervised learning in classification.
Revised: Since |X| < + ∞ , we know that |H| < + ∞ , which implies that H in is agnostic PAC learnable for supervised learning in classification.

Original: Therefore, there exist an algorithm
Revised: Therefore, there exist an algorithm

Original: A in : ∪ + ∞ n =1 ( X × Y ) n → H in and a monotonically decreasing sequence ϵ ( n ) , such that ϵ ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D sXY ,
Revised: A in : ∪ + ∞ n =1 ( X × Y ) n → H in and a monotonically decreasing sequence ϵ ( n ) , such that ϵ ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D sXY ,

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation]
Revised: [Equation]

Original: A in ( S ))
Revised: A in ( S ))

Original: [Equation]
Revised: [Equation]

Original: Since |X| < + ∞ and H b almost contains all binary classifiers, then using Theorem 6 and Theorem 1, we obtain that there exist an algorithm A b : ∪ + ∞ n =1 ( X × { 1 , 2 } ) n → H b and a monotonically decreasing sequence ϵ ′ ( n ) , such that ϵ ′ ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D sXY ,
Revised: Since |X| < + ∞ and H b almost contains all binary classifiers, then using Theorem 6 and Theorem 1, we obtain that there exist an algorithm A b : ∪ + ∞ n =1 ( X × { 1 , 2 } ) n → H b and a monotonically decreasing sequence ϵ ′ ( n ) , such that ϵ ′ ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D sXY ,

Original: [Equation]
Revised: [Equation]

Original: where ϕ maps ID’s labels to 1 and OOD’s label to 2 ,
Revised: where ϕ maps ID’s labels to 1 and OOD’s label to 2 ,

Original: [Equation]
Revised: [Equation]

Original: ) and
Revised: ) and

Original: [Equation]
Revised: [Equation]

Original: 39) here ϕ ( S ) =
Revised: 39) here ϕ ( S ) =

Original: { ( x 1 , ϕ
Revised: { ( x 1 , ϕ

Original: ( y 1 )) , ..., ( x n , ϕ ( y n )) } , if S = { ( x
Revised: ( y 1 )) , ..., ( x n , ϕ ( y n )) } , if S = { ( x

Original: [Equation]
Revised: [Equation]

Original: , y
Revised: , y

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ..., ( x n , y n ) } .
Revised: ..., ( x n , y n ) } .

Original: Note that H b almost contains all classifiers, and D s XY is the separate space.
Revised: Note that H b almost contains all classifiers, and D s XY is the separate space.

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: Next , we construct an algorithm A using A in and A out .
Revised: Next , we construct an algorithm A using A in and A out .

Original: [Equation]
Revised: [Equation]

Original: Since inf h ∈H R in ϕ ( D ) ( ϕ ◦ h )
Revised: Since inf h ∈H R in ϕ ( D ) ( ϕ ◦ h )

Original: = 0 , inf h ∈H R out D ( h ) = 0 , then by Condition 2, it is easy to check that
Revised: = 0 , inf h ∈H R out D ( h ) = 0 , then by Condition 2, it is easy to check that

Original: [Equation]
Revised: [Equation]

Original: h
Revised: h

Original: [Equation]
Revised: [Equation]

Original: in R in
Revised: in R in

Original: [Equation]
Revised: [Equation]

Original: h ) .
Revised: h ) .

Original: Additionally, the risk R in D ( A ( S )) is from two parts: 1) ID data are detected as OOD data; 2) ID data are detected as ID data, but are classified as incorrect ID classes.
Revised: Additionally, the risk R in D ( A ( S )) is from two parts: 1) ID data are detected as OOD data; 2) ID data are detected as ID data, but are classified as incorrect ID classes.

Original: Therefore, we have the inequality:
Revised: Therefore, we have the inequality:

Original: [Equation]
Revised: [Equation]

Original: n
Revised: n

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where c = max y 1 ,y 2 ∈Y ℓ ( y 1 , y 2 ) / min { ℓ (1 , 2) , ℓ (2 , 1) } .
Revised: where c = max y 1 ,y 2 ∈Y ℓ ( y 1 , y 2 ) / min { ℓ (1 , 2) , ℓ (2 , 1) } .

Original: Note that the risk R out D ( A ( S )) is from the case that OOD data are detected as ID data.
Revised: Note that the risk R out D ( A ( S )) is from the case that OOD data are detected as ID data.

Original: Therefore,
Revised: Therefore,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Note that (1 − α ) inf h ∈H R in D ( h ) + α inf h ∈H R out D ( h ) ≤ inf h ∈H R αD ( h ) .
Revised: Note that (1 − α ) inf h ∈H R in D ( h ) + α inf h ∈H R out D ( h ) ≤ inf h ∈H R αD ( h ) .

Original: Then, using Eq.
Revised: Then, using Eq.

Original: (40) and
Revised: (40) and

Original: Eq.
Revised: Eq.

Original: (41), we obtain that for any α ∈ [0 , 1] ,
Revised: (41), we obtain that for any α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: According to Theorem 1 (the second result), we complete the proof.
Revised: According to Theorem 1 (the second result), we complete the proof.

Original: J Proofs of Theorems 8 andJ.1 Proof of Theorem
Revised: J Proofs of Theorems 8 andJ.1 Proof of Theorem

Original: Lemma 8.
Revised: Lemma 8.

Original: Given a prior-unknown space D XY and a hypothesis space H , if Condition 3 holds, then for any equivalence class [ D ′ XY ] with respect to D XY , OOD detection is learnable in the equivalence class [ D ′ XY ] for H .
Revised: Given a prior-unknown space D XY and a hypothesis space H , if Condition 3 holds, then for any equivalence class [ D ′ XY ] with respect to D XY , OOD detection is learnable in the equivalence class [ D ′ XY ] for H .

Original: Furthermore, the learning rate can attain O (1 /n ) .
Revised: Furthermore, the learning rate can attain O (1 /n ) .

Original: Proof.
Revised: Proof.

Original: Let F be a set consisting of all infinite sequences, whose coordinates are hypothesis functions, i.e. ,
Revised: Let F be a set consisting of all infinite sequences, whose coordinates are hypothesis functions, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: For each h ∈ F , there is a corresponding algorithm
Revised: For each h ∈ F , there is a corresponding algorithm

Original: A h : A h ( S ) =
Revised: A h : A h ( S ) =

Original: h n , if | S | = n .
Revised: h n , if | S | = n .

Original: F generates an algorithm class A
Revised: F generates an algorithm class A

Original: = { A h : ∀ h ∈ F } .
Revised: = { A h : ∀ h ∈ F } .

Original: We select a consistent algorithm from the algorithm class A .
Revised: We select a consistent algorithm from the algorithm class A .

Original: We construct a special infinite sequence ˜ h
Revised: We construct a special infinite sequence ˜ h

Original: = (˜ h 1 , ..., ˜ h n , ... ) ∈ F .
Revised: = (˜ h 1 , ..., ˜ h n , ... ) ∈ F .

Original: For each positive integer n , we select ˜ h n from (cid:92)
Revised: For each positive integer n , we select ˜ h n from (cid:92)

Original: [Equation]
Revised: [Equation]

Original: The existence of ˜ h n is based on Condition 3.
Revised: The existence of ˜ h n is based on Condition 3.

Original: It is easy to check that for any D XY ∈
Revised: It is easy to check that for any D XY ∈

Original: [ D ′ XY ] ,
Revised: [ D ′ XY ] ,

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: /n
Revised: /n

Original: [Equation]
Revised: [Equation]

Original: Since (1 − α ) inf h ∈H R in D ( h )+ α inf h ∈H R out D ( h ) ≤ inf h ∈H R αD ( h ) , we obtain that for any α ∈ [0 , 1] ,
Revised: Since (1 − α ) inf h ∈H R in D ( h )+ α inf h ∈H R out D ( h ) ≤ inf h ∈H R αD ( h ) , we obtain that for any α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: Using Theorem 1 (the second result), we have completed this proof.
Revised: Using Theorem 1 (the second result), we have completed this proof.

Original: Theorem 8.
Revised: Theorem 8.

Original: Suppose that X is a bounded set.
Revised: Suppose that X is a bounded set.

Original: OOD detection is learnable in the finite-ID-distribution space D FXY for H if and only if the compatibility condition (i.e., Condition 3) holds.
Revised: OOD detection is learnable in the finite-ID-distribution space D FXY for H if and only if the compatibility condition (i.e., Condition 3) holds.

Original: Furthermore, the learning rate ϵ cons ( n ) can attain O (1 / √ n 1 − θ ) , for any θ ∈ (0 , 1) .
Revised: Furthermore, the learning rate ϵ cons ( n ) can attain O (1 / √ n 1 − θ ) , for any θ ∈ (0 , 1) .

Original: Proof of Theorem 8.
Revised: Proof of Theorem 8.

Original: First , we prove that if OOD detection is learnable in D FXY for H , then Condition 3 holds.
Revised: First , we prove that if OOD detection is learnable in D FXY for H , then Condition 3 holds.

Original: Since D FXY is the prior-unknown space, by Theorem 1, there exist an algorithm A : ∪ + ∞ n =1 ( X ×Y ) n →
Revised: Since D FXY is the prior-unknown space, by Theorem 1, there exist an algorithm A : ∪ + ∞ n =1 ( X ×Y ) n →

Original: H and a monotonically decreasing sequence ϵ cons ( n ) , such that ϵ cons ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D FXY ,
Revised: H and a monotonically decreasing sequence ϵ cons ( n ) , such that ϵ cons ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D FXY ,

Original: [Equation]
Revised: [Equation]

Original: Then, for any ϵ > 0 , we can find n ϵ such that ϵ ≥ ϵ cons ( n ϵ ) , therefore, if n = n ϵ , we have
Revised: Then, for any ϵ > 0 , we can find n ϵ such that ϵ ≥ ϵ cons ( n ϵ ) , therefore, if n = n ϵ , we have

Original: [Equation]
Revised: [Equation]

Original: which implies that there exists S ϵ ∼ D n ϵ X
Revised: which implies that there exists S ϵ ∼ D n ϵ X

Original: I Y I such that
Revised: I Y I such that

Original: [Equation]
Revised: [Equation]

Original: Therefore, for any equivalence class [ D ′ XY ] with respect to D FXY and any ϵ > 0 , there exists a hypothesis function A ( S ϵ ) ∈ H such that for any domain D XY ∈
Revised: Therefore, for any equivalence class [ D ′ XY ] with respect to D FXY and any ϵ > 0 , there exists a hypothesis function A ( S ϵ ) ∈ H such that for any domain D XY ∈

Original: [ D ′ XY ] ,
Revised: [ D ′ XY ] ,

Original: [Equation]
Revised: [Equation]

Original: which implies that Condition 3 holds.
Revised: which implies that Condition 3 holds.

Original: Second , we prove Condition 3 implies the learnability of OOD detection in D FXY for H .
Revised: Second , we prove Condition 3 implies the learnability of OOD detection in D FXY for H .

Original: For convenience, we assume that all equivalence classes are [ D 1 XY ] , ..., [ D mXY ] .
Revised: For convenience, we assume that all equivalence classes are [ D 1 XY ] , ..., [ D mXY ] .

Original: By Lemma 8, for every equivalence class [ D iXY ] , we can find a corresponding algorithm
Revised: By Lemma 8, for every equivalence class [ D iXY ] , we can find a corresponding algorithm

Original: A D
Revised: A D

Original: i such that OOD detection is learnable in [ D iXY ] for H .
Revised: i such that OOD detection is learnable in [ D iXY ] for H .

Original: Additionally, we also set the learning rate for A D i is ϵ i ( n ) .
Revised: Additionally, we also set the learning rate for A D i is ϵ i ( n ) .

Original: By Lemma 8, we know that ϵ i ( n ) can attain O (1 /n ) .
Revised: By Lemma 8, we know that ϵ i ( n ) can attain O (1 /n ) .

Original: Let Z be X ×Y .
Revised: Let Z be X ×Y .

Original: Then, we consider a bounded universal kernel K ( · , · ) defined over Z ×Z .
Revised: Then, we consider a bounded universal kernel K ( · , · ) defined over Z ×Z .

Original: Consider the maximum mean discrepancy (MMD) [77], which is a metric between distributions: for any distributions P and Q defined over Z , we use MMD K ( Q, P ) to represent the distance.
Revised: Consider the maximum mean discrepancy (MMD) [83], which is a metric between distributions: for any distributions P and Q defined over Z , we use MMD K ( Q, P ) to represent the distance.

Original: Let F be a set consisting of all finite sequences, whose coordinates are labeled data, i.e. ,
Revised: Let F be a set consisting of all finite sequences, whose coordinates are labeled data, i.e. ,

Original: F = { S =
Revised: F = { S =

Original: ( S 1 , ..., S i , ..., S m ) : ∀ i = 1 , ..., m and ∀ labeled data S i } .
Revised: ( S 1 , ..., S i , ..., S m ) : ∀ i = 1 , ..., m and ∀ labeled data S i } .

Original: Then, we define an algorithm space as follows:
Revised: Then, we define an algorithm space as follows:

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: here
Revised: here

Original: [Equation]
Revised: [Equation]

Original: and δ ( x ,y ) is the Dirac measure.
Revised: and δ ( x ,y ) is the Dirac measure.

Original: Next, we prove that we can find an algorithm A from the algorithm space
Revised: Next, we prove that we can find an algorithm A from the algorithm space

Original: A such that A is the consistent algorithm.
Revised: A such that A is the consistent algorithm.

Original: Since the number of different equivalence classes is finite, we know that there exists a constant c >such that for any different equivalence classes [ D iXY ] and [ D jXY ] ( i ̸ = j ),
Revised: Since the number of different equivalence classes is finite, we know that there exists a constant c >such that for any different equivalence classes [ D iXY ] and [ D jXY ] ( i ̸ = j ),

Original: [Equation]
Revised: [Equation]

Original: Additionally, according to [77] and the property of D FXY (the number of different equivalence classes is finite), there exists a monotonically decreasing ϵ ( n ) → 0 , as n → + ∞ such that for any D XY ∈ D ,
Revised: Additionally, according to [83] and the property of D FXY (the number of different equivalence classes is finite), there exists a monotonically decreasing ϵ ( n ) → 0 , as n → + ∞ such that for any D XY ∈ D ,

Original: [Equation]
Revised: [Equation]

Original: , where
Revised: , where

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Therefore, for every equivalence class [ D iXY ] , we can find data points S D i such that
Revised: Therefore, for every equivalence class [ D iXY ] , we can find data points S D i such that

Original: [Equation]
Revised: [Equation]

Original: Let S ′
Revised: Let S ′

Original: = { S D 1 , ..., S D i , ..., S D m } .
Revised: = { S D 1 , ..., S D i , ..., S D m } .

Original: Then, we prove that A S ′ is a consistent algorithm.
Revised: Then, we prove that A S ′ is a consistent algorithm.

Original: By Eq.
Revised: By Eq.

Original: (42), it is easy to check that for any i ∈ { 1 , ..., m } and any 0 < δ < 1 ,
Revised: (42), it is easy to check that for any i ∈ { 1 , ..., m } and any 0 < δ < 1 ,

Original: [Equation]
Revised: [Equation]

Original: , which implies that
Revised: , which implies that

Original: [Equation]
Revised: [Equation]

Original: Therefore, (here we set δ = 200 ϵ ( n ) /c )
Revised: Therefore, (here we set δ = 200 ϵ ( n ) /c )

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: c .
Revised: c .

Original: Because A D i is a consistent algorithm for [ D iXY ] , we conclude that for all α ∈ [0 , 1] ,
Revised: Because A D i is a consistent algorithm for [ D iXY ] , we conclude that for all α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: where ϵ i ( n ) =
Revised: where ϵ i ( n ) =

Original: O (1 /n ) is the learning rate of A D i and B is the upper bound of the loss ℓ .
Revised: O (1 /n ) is the learning rate of A D i and B is the upper bound of the loss ℓ .

Original: Let ϵ max ( n ) =
Revised: Let ϵ max ( n ) =

Original: max { ϵ 1 ( n ) , ..., ϵ m ( n ) } + 200 Bϵ ( n ) c .
Revised: max { ϵ 1 ( n ) , ..., ϵ m ( n ) } + 200 Bϵ ( n ) c .

Original: Then, we obtain that for any D XY ∈ D FXY and all α ∈ [0 , 1] ,
Revised: Then, we obtain that for any D XY ∈ D FXY and all α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: According to Theorem 1 (the second result), A S ′ is the consistent algorithm.
Revised: According to Theorem 1 (the second result), A S ′ is the consistent algorithm.

Original: This proof is completed.
Revised: This proof is completed.

Original: J.2 Proof of Theorem
Revised: J.2 Proof of Theorem

Original: Theorem 9.
Revised: Theorem 9.

Original: Given a density-based space D µ,bXY , if µ ( X ) < + ∞ , the Realizability Assumption holds, then when H has finite Natarajan dimension [21], OOD detection is learnable in D µ,bXY for H .
Revised: Given a density-based space D µ,bXY , if µ ( X ) < + ∞ , the Realizability Assumption holds, then when H has finite Natarajan dimension [21], OOD detection is learnable in D µ,bXY for H .

Original: Furthermore, the learning rate ϵ cons ( n ) can attain O (1 / √ n 1 − θ ) , for any θ ∈ (0 , 1) .
Revised: Furthermore, the learning rate ϵ cons ( n ) can attain O (1 / √ n 1 − θ ) , for any θ ∈ (0 , 1) .

Original: Proof of Theorem 9.
Revised: Proof of Theorem 9.

Original: First , we consider the case that the loss ℓ is the zero-one loss.
Revised: First , we consider the case that the loss ℓ is the zero-one loss.

Original: Since µ ( X ) < + ∞ , without loss of generality, we assume that µ ( X ) = 1 .
Revised: Since µ ( X ) < + ∞ , without loss of generality, we assume that µ ( X ) = 1 .

Original: We also assume that f I is D X
Revised: We also assume that f I is D X

Original: I ’s density function and f O is D
Revised: I ’s density function and f O is D

Original: X
Revised: X

Original: O ’s density function.
Revised: O ’s density function.

Original: Let f be the density function for 0 .
Revised: Let f be the density function for 0 .

Original: 5 ∗ D X
Revised: 5 ∗ D X

Original: I + 0 .
Revised: I + 0 .

Original: 5 ∗ D X O .
Revised: 5 ∗ D X O .

Original: It is easy to check that f = 0 .
Revised: It is easy to check that f = 0 .

Original: 5 ∗ f
Revised: 5 ∗ f

Original: I + 0 .
Revised: I + 0 .

Original: 5 ∗ f O .
Revised: 5 ∗ f O .

Original: Additionally, due to
Revised: Additionally, due to

Original: Realizability Assumption, it is obvious that for any samples S
Revised: Realizability Assumption, it is obvious that for any samples S

Original: = { ( x 1 , y 1 ) , ..., ( x n , y n ) } ∼ D nX
Revised: = { ( x 1 , y 1 ) , ..., ( x n , y n ) } ∼ D nX

Original: I Y
Revised: I Y

Original: I , i.i.d., we have that there exists h ∗ ∈ H such that
Revised: I , i.i.d., we have that there exists h ∗ ∈ H such that

Original: [Equation]
Revised: [Equation]

Original: Given m data points S m = { x ′ 1 , ..., x ′ m } ⊂ X m .
Revised: Given m data points S m = { x ′ 1 , ..., x ′ m } ⊂ X m .

Original: We consider the following learning rule:
Revised: We consider the following learning rule:

Original: [Equation]
Revised: [Equation]

Revised: subject

Revised: [Equation]

Original: We denote the algorithm, which solves the above rule, as A S m  .
Revised: We denote the algorithm, which solves the above rule, as A S m 8 .

Original: For different data points S m , we have different algorithm A S m .
Revised: For different data points S m , we have different algorithm A S m .

Original: Let S be the infinite sequence set that consists of all infinite sequences, whose coordinates are data points, i.e. ,S : [Equation] { S
Revised: Let S be the infinite sequence set that consists of all infinite sequences, whose coordinates are data points, i.e. ,

Original: [Equation]
Revised: [Equation]

Original: S

Original: [Equation] S 2 , ..., Sm [Equation] ... ) : S m are any m data points , m = 1 , ...
Revised: Using S , we construct an algorithm space as follows:

Original: [Equation]
Revised: [Equation]

Original: + ∞} . [Equation] Using S , we construct an algorithm space as follows:
Revised: A

Original: [Equation]
Revised: [Equation]

Original: Next, we prove that there exists an algorithm A S ∈ A , which is a consistent algorithm.
Revised: Next, we prove that there exists an algorithm A S ∈ A , which is a consistent algorithm.

Original: Given data points S n ∼ µ n , i.i.d., using the Natarajan dimension theory and Empirical risk minimization principle [21], it is easy to obtain that there exists a uniform constant C θ such that (we mainly use the uniform bounds to obtain the following bounds)
Revised: Given data points S n ∼ µ n , i.i.d., using the Natarajan dimension theory and Empirical risk minimization principle [21], it is easy to obtain that there exists a uniform constant C θ such that (we mainly use the uniform bounds to obtain the following bounds)

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: 44) where
Revised: ) where

Original: H S =
Revised: H S =

Original: { h ∈ H : n (cid:88) i =
Revised: { h ∈ H : n [Equation]

Original:  ℓ ( h ( x i ) , y i ) = 0 } , here S=
Revised: i = ℓ ( h ( x i ) , y i ) = 0 } , here S

Original:  { ( x 1 , y 1 ) , ..., ( x n , y n ) } ∼ D nX
Revised: = { ( x 1 , y 1 ) , ..., ( x n , y n ) } ∼ D nX

Original: I Y I , and
Revised: I Y I , and

Original: [Equation]
Revised: [Equation]

Original: Due to Realizability Assumption, we obtain that inf h ∈H R in D ( h ) = 0 .
Revised: Due to Realizability Assumption, we obtain that inf h ∈H R in D ( h ) = 0 .

Original: Therefore,
Revised: Therefore,

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: which implies that (in following inequalities, g is the groundtruth labeling function,
Revised: which implies that (in following inequalities, g is the groundtruth labeling function,

Original: i.e. , R D ( g ) = 0 )
Revised: i.e. , R D ( g ) = 0 )

Original: [Equation]
Revised: [Equation]

Original: This implies that (here we have used the property of zero-one loss)
Revised: This implies that (here we have used the property of zero-one loss)

Original: [Equation]
Revised: [Equation]

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: Additionally, R µ ( g, K + [Equation]
Revised: Additionally, R µ ( g, K + 1) = µ ( x ∈ X : g ( x ) <

Original: = µ ( x ∈ X : g ( x ) < K + 1) and g ∈ H S , which implies that
Revised: K + 1) and g ∈ H S , which implies that

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Combining inequalities (46) and (47), we obtain that
Revised: Combining inequalities (46) and (47), we obtain that

Original: [Equation]
Revised: [Equation]

Original: Using inequalities (44) and (48)
Revised: Using inequalities (44) and (48)

Original: [Equation]
Revised: [Equation]

Original: we obtain that
Revised: we obtain that

Original: [Equation]
Revised: [Equation]

Original: By Fubini theorem
Revised: By Fubini theorem

Original: [Equation]
Revised: [Equation]

Original: we have that
Revised: we have that

Original: [Equation]
Revised: [Equation]

Original: Using inequality (45)
Revised: Using inequality (45)

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: [Equation] which implies that (here we use the property of zero-one loss) 
Revised: which implies that (here we use the property of zero-one loss) [Equation]

Revised: I

Original: [Equation]
Revised: [Equation]

Original: Combining inequalities (49) and (51), we have
Revised: Combining inequalities (49) and (51), we have

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: there exist data points S ′ n such that
Revised: there exist data points S ′ n such that

Original: [Equation]
Revised: [Equation]

Original: Combining inequalities (45) and (52) [Equation] we obtain that for any n , there exists data points S ′ n such that
Revised: Combining inequalities (45) and (52), we obtain that for any n , there exists data points S ′ n such that

Original: [Equation]
Revised: [Equation]

Original: We set data point sequences S ′
Revised: We set data point sequences S ′

Original: =
Revised: =

Original: ( S ′ 1 , S ′ 2 , ..., S ′ n , ... ) .
Revised: ( S ′ 1 , S ′ 2 , ..., S ′ n , ... ) .

Original: Then, A S ′ ∈
Revised: Then, A S ′ ∈

Original: A is the universally consistent algorithm, i
Revised: A is the universally consistent algorithm, i.e. , for any α ∈ [0 , 1]

Original: [Equation]
Revised: [Equation]

Original: e. , for any α ∈ [0 , 1]

Original: [Equation]
Revised: [Equation]

Original: θ (cid:9) .

Original: We have completed this proof when ℓ is the zero-one loss.

Original: Second , we prove the case that ℓ is not the zero-one loss.
Revised: Second , we prove the case that ℓ is not the zero-one loss.

Original: We use the notation ℓ 0 − 1 as the zero-one loss.
Revised: We use the notation ℓ 0 − 1 as the zero-one loss.

Original: According the definition of loss introduced in Section 2, we know that there exists a constant
Revised: According the definition of loss introduced in Section 2, we know that there exists a constant

Original: M > 0 such that for any y 1 , y 2 ∈ Y all ,
Revised: M > 0 such that for any y 1 , y 2 ∈ Y all ,

Original: [Equation]
Revised: [Equation]

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: where R α,ℓ 0 − 1 D is the α -risk with zero-one loss, and R α,ℓD is the α -risk for loss ℓ .
Revised: where R α,ℓ [Equation] 1 D is the α -risk with zero-one loss, and R α,ℓD is the α -risk for loss ℓ .

Original: Above inequality tells us that Realizability Assumption holds with zero-one loss if and only if
Revised: Above inequality tells us that Realizability Assumption holds with zero-one loss if and only if

Original: Realizability Assumption holds with the loss ℓ .
Revised: Realizability Assumption holds with the loss ℓ .

Original: Therefore, we use the result proven in first step.
Revised: Therefore, we use the result proven in first step.

Original: We can find a consistent algorithm
Revised: We can find a consistent algorithm

Original: A such that for any α ∈ [0 , 1] ,
Revised: A such that for any α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: which implies that for any α ∈ [0 , 1] ,
Revised: which implies that for any α ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: We have completed this proof.
Revised: We have completed this proof.

Original: K Proof of Proposition 1 and Proof of Proposition
Revised: K Proof of Proposition 1 and Proof of Proposition

Original: To better understand the contents in Appendices K-M, we introduce the important notations for
Revised: To better understand the contents in Appendices K-M, we introduce the important notations for

Original: FCNN-based hypothesis space and score-based hypothesis space detaily.
Revised: FCNN-based hypothesis space and score-based hypothesis space detaily.

Original: FCNN-based Hypothesis Space.
Revised: FCNN-based Hypothesis Space.

Original: Given a sequence q
Revised: Given a sequence q

Original: =
Revised: =

Original: ( l 1 , l 2 , ..., l g ) , where l i and g are positive integers and g > 2 , we use g to represent the depth of neural network and use l i to represent the width of the i -th layer.
Revised: ( l 1 , l 2 , ..., l g ) , where l i and g are positive integers and g > 2 , we use g to represent the depth of neural network and use l i to represent the width of the i -th layer.

Original: After the activation function σ is selected, we can obtain the architecture of
Revised: After the activation function σ is selected, we can obtain the architecture of

Original: FCNN according to the sequence q .
Revised: FCNN according to the sequence q .

Original: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the output of the i -layer can be written as follows: for any x ∈ R l 1 ,
Revised: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the output of the i -layer can be written as follows: for any x ∈ R l 1 ,

Original: [Equation]
Revised: [Equation]

Original: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .
Revised: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .

Original: Then, the output of FCNN is f w , b ( x ) =
Revised: Then, the output of FCNN is f w , b ( x ) =

Original: w g f g − 1 ( x ) + b g , where w = { w 2 , ..., w g } and b = { b 2 , ..., b g } .
Revised: w g f g − 1 ( x ) + b g , where w = { w 2 , ..., w g } and b = { b 2 , ..., b g } .

Original: An FCNN-based scoring function space is defined as:
Revised: An FCNN-based scoring function space is defined as:

Original: [Equation]
Revised: [Equation]

Original: Additionally, given two sequences q = ( l 1 , ..., l g ) and q ′ =
Revised: Additionally, given two sequences q = ( l 1 , ..., l g ) and q ′ =

Original: ( l ′ 1 , ..., l ′ g ′ ) , we use the notation q ≲ q ′ to represent the following equations and inequalities:
Revised: ( l ′ 1 , ..., l ′ g ′ ) , we use the notation q ≲ q ′ to represent the following equations and inequalities:

Original: [Equation]
Revised: [Equation]

Original: Given a sequence q = ( l 1 , ...
Revised: Given a sequence q = ( l 1 , ...

Original: l g ) satisfying that l 1 = d and l g = K + 1 , the FCNN-based scoring function space F σ q can induce an FCNN-based hypothesis space.
Revised: l g ) satisfying that l 1 = d and l g = K + 1 , the FCNN-based scoring function space F σ q can induce an FCNN-based hypothesis space.

Original: Before defining the FCNN-based hypothesis space, we define the induced hypothesis function.
Revised: Before defining the FCNN-based hypothesis space, we define the induced hypothesis function.

Original: For any f w , b ∈ F σ q , the induced hypothesis function is:
Revised: For any f w , b ∈ F σ q , the induced hypothesis function is:

Original: [Equation]
Revised: [Equation]

Original: where f k w , b ( x ) is the k -th coordinate of f w , b ( x ) .
Revised: where f k w , b ( x ) is the k -th coordinate of f w , b ( x ) .

Original: Then, we define the FCNN-based hypothesis space as follows:
Revised: Then, we define the FCNN-based hypothesis space as follows:

Original: [Equation]
Revised: [Equation]

Revised: l g

Original: Score-based Hypothesis Space.
Revised: Score-based Hypothesis Space.

Original: Many OOD algorithms detect OOD data using a score-based strategy.
Revised: Many OOD algorithms detect OOD data using a score-based strategy.

Original: That is, given a threshold λ , a scoring function space F l ⊂ { f : X → R l } and a scoring function E : F l → R , then x is regarded as ID, if E ( f ( x )) ≥ λ ; otherwise, x is regarded as OOD.
Revised: That is, given a threshold λ , a scoring function space F l ⊂ { f : X → R l } and a scoring function E : F l → R , then x is regarded as ID, if E ( f ( x )) ≥ λ ; otherwise, x is regarded as OOD.

Original: Using E , λ and f ∈ F σ q , we can generate a binary classifier h λ f ,E : 
Revised: Using E , λ and f ∈ F σ q , we can generate a binary classifier h λ f ,E : h

Revised: λ f ,E ( x ) := (cid:26)

Revised: [Equation]

Revised: , if E ( f ( x )) ≥ λ

Original: [Equation]
Revised: [Equation]

Revised: 2 , if E ( f ( x )) <

Original:  where 1 represents ID data, and 2 represents OOD data.
Revised: λ, where 1 represents ID data, and 2 represents OOD data.

Original: Hence, a binary classification hypothesis space H b , which consists of all h λ f ,E , is generated.
Revised: Hence, a binary classification hypothesis space H b , which consists of all h λ f ,E , is generated.

Original: We define the score-based hypothesis space
Revised: We define the score-based hypothesis space

Original: H σ,λ q ,E := { h λ f ,E : ∀ f ∈ F σ q } .
Revised: H σ,λ q ,E := { h λ f ,E : ∀ f ∈ F σ q } .

Original: Next, we introduce two important propositions.
Revised: Next, we introduce two important propositions.

Original: Proposition 1.
Revised: Proposition 1.

Original: Given a sequence q = ( l 1 , ...
Revised: Given a sequence q = ( l 1 , ...

Original: l g ) satisfying that l 1 = d and l g = K + 1 (note that d is the dimension of input data and K + 1 is the dimension of output), then the constant functions h 1 , h 2 ,..., h K +1 belong to H σ q , where h i ( x )
Revised: l g ) satisfying that l 1 = d and l g = K + 1 (note that d is the dimension of input data and K + 1 is the dimension of output), then the constant functions h 1 , h 2 ,..., h K +1 belong to H σ q , where h i ( x )

Original: =
Revised: =

Original: i , for any x ∈ X .
Revised: i , for any x ∈ X .

Original: Therefore, Assumption 1 holds for H σ q .
Revised: Therefore, Assumption 1 holds for H σ q .

Original: Proof of Proposition 1.
Revised: Proof of Proposition 1.

Original: Note that the output of FCNN can be written as
Revised: Note that the output of FCNN can be written as

Original: [Equation]
Revised: [Equation]

Original: where w g ∈ R ( K +1)
Revised: where w g ∈ R ( K +1)

Original: × l g − 1 , b g ∈ R ( K +1) × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.
Revised: × l g − 1 , b g ∈ R ( K +1) × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.

Original: If we set w g = 0 , and set b g = y i , where y i is the one-hot vector corresponding to label i .
Revised: If we set w g = 0 , and set b g = y i , where y i is the one-hot vector corresponding to label i .

Original: Then f w , b ( x ) =
Revised: Then f w , b ( x ) =

Original: y i , for any x ∈ X .
Revised: y i , for any x ∈ X .

Original: Therefore, h i ( x ) ∈ H σ q , for any i
Revised: Therefore, h i ( x ) ∈ H σ q , for any i

Original: = 1 , ..., K, K + 1 .
Revised: = 1 , ..., K, K + 1 .

Original: Note that in some works [78], b g is fixed to 0 .
Revised: Note that in some works [84], b g is fixed to 0 .

Original: In fact, it is easy to check that when g > 2 and activation function σ is not a constant, Proposition 1 still holds, even if b g = 0 .
Revised: In fact, it is easy to check that when g > 2 and activation function σ is not a constant, Proposition 1 still holds, even if b g = 0 .

Original: Proposition 2.
Revised: Proposition 2.

Original: For any sequence q = ( l 1 , ..., l g ) satisfying that l 1 = d and l g = l ( note that d is the dimension of input data and l is the dimension of output ) , if { v ∈ R l : E ( v ) ≥ λ } ̸ = ∅ and { v ∈ R l : E ( v ) < λ } ̸ = ∅ , then the functions h 1 and h 2 belong to H σ,λ q ,E , where h 1 ( x ) = 1 and h 2 ( x )
Revised: For any sequence q = ( l 1 , ..., l g ) satisfying that l 1 = d and l g = l ( note that d is the dimension of input data and l is the dimension of output ) , if { v ∈ R l : E ( v ) ≥ λ } ̸ = ∅ and { v ∈ R l : E ( v ) < λ } ̸ = ∅ , then the functions h 1 and h 2 belong to H σ,λ q ,E , where h 1 ( x ) = 1 and h 2 ( x )

Original: = 2 , for any x ∈ X , where 1 represents the ID labels, and 2 represents the OOD labels.
Revised: = 2 , for any x ∈ X , where 1 represents the ID labels, and 2 represents the OOD labels.

Original: Therefore, Assumption 1 holds.
Revised: Therefore, Assumption 1 holds.

Original: Proof of Proposition 2.
Revised: Proof of Proposition 2.

Original: Since { v ∈ R l : E ( v ) ≥ λ } ̸ = ∅ and { v ∈ R l : E ( v ) < λ } ̸ = ∅ , we can find v 1 ∈ { v ∈ R l : E ( v ) ≥ λ } and v 2 ∈ { v ∈ R l : E ( v ) < λ } .
Revised: Since { v ∈ R l : E ( v ) ≥ λ } ̸ = ∅ and { v ∈ R l : E ( v ) < λ } ̸ = ∅ , we can find v 1 ∈ { v ∈ R l : E ( v ) ≥ λ } and v 2 ∈ { v ∈ R l : E ( v ) < λ } .

Original: For any f w , b ∈ F σ q , we have [Equation]
Revised: For any f w , b ∈ F σ q , we have f w , b ( x ) =

Original:  where w g ∈ R l × l g − 1 , b g ∈ R l × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.
Revised: w g f g − 1 ( x ) + b g , where w g ∈ R l × l g − 1 , b g ∈ R l × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.

Original: If we set w g = 

Revised: If we set w g =

Original: l × l g − 1 and b g = v 1 , then f w , b ( x ) = v 1 for any x ∈ X , where 0 l × l g − 1 is l × l g −zero matrix.
Revised: l × l g − 1 and b g = v 1 , then f w , b ( x ) = v 1 for any x ∈ X , where 0 l × l g − 1 is l × l g −zero matrix.

Original: Hence, h 1 can be induced by f w , b .
Revised: Hence, h 1 can be induced by f w , b .

Original: Therefore, h 1 ∈ H σ,λ q ,E .
Revised: Therefore, h 1 ∈ H σ,λ q ,E .

Original: Similarly, if we set w g = 

Revised: Similarly, if we set w g =

Original: l × l g − 1 and b g = v 2 , then f w , b ( x ) = v 2 for any x ∈ X , where 0 l × l g − 1 is l × l g − 1 zero matrix.
Revised: l × l g − 1 and b g = v 2 , then f w , b ( x ) = v 2 for any x ∈ X , where 0 l × l g − 1 is l × l g − 1 zero matrix.

Original: Hence, h 2 can be induced by f w , b .
Revised: Hence, h 2 can be induced by f w , b .

Original: Therefore, h 2 ∈ H σ,λ q ,E .
Revised: Therefore, h 2 ∈ H σ,λ q ,E .

Original: It is easy to check that when g > 2 and activation function σ is not a constant, Proposition 2 still holds, even if b g = 0 .
Revised: It is easy to check that when g > 2 and activation function σ is not a constant, Proposition 2 still holds, even if b g = 0 .

Original: L Proof of Theorem
Revised: L Proof of Theorem

Original: Before proving Theorem 10, we need several lemmas.
Revised: Before proving Theorem 10, we need several lemmas.

Original: Lemma 9.
Revised: Lemma 9.

Original: Let σ be ReLU function: max { x, 0 } .
Revised: Let σ be ReLU function: max { x, 0 } .

Original: Given q = ( l 1 , ..., l g ) and q ′ =
Revised: Given q = ( l 1 , ..., l g ) and q ′ =

Original: ( l ′ 1 , ..., l ′ g ) such that l g = l ′ g and l 1 = l ′ 1 , and l i ≤ l ′
Revised: ( l ′ 1 , ..., l ′ g ) such that l g = l ′ g and l 1 = l ′ 1 , and l i ≤ l ′

Original: i
Revised: i

Original: ( i = 1 , ..., g − 1) , then F σ q ⊂ F σ q ′ and H σ q ⊂ H σ q ′ .
Revised: ( i = 1 , ..., g − 1) , then F σ q ⊂ F σ q ′ and H σ q ⊂ H σ q ′ .

Original: Proof of Lemma 9.
Revised: Proof of Lemma 9.

Original: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the i -layer output of
Revised: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the i -layer output of

Original: FCNN with architecture q can be written as
Revised: FCNN with architecture q can be written as

Original: [Equation]
Revised: [Equation]

Original: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .
Revised: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .

Original: Then, the output of last layer is
Revised: Then, the output of last layer is

Original: [Equation]
Revised: [Equation]

Revised: b

Revised: [Equation]

Revised: x

Revised: [Equation]

Original:  We will show that f w , b ∈ F σ q ′ .
Revised: = [Equation] + b g . We will show that f w , b ∈ F σ q ′ .

Original: We construct f w ′ , b
Revised: We construct f w ′ , b

Original: as follows: for every w ′ i ∈ R l ′ i × l ′ i − 1 , if l ′ i − l i >and l ′ i − 1 − l i − 1 > 0 , we set
Revised: as follows: for every w ′ i ∈ R l [Equation] i × l ′ i − 1 , if l ′ i − l i >and l ′ i − 1 − l i − 1 > 0 , we set

Original: [Equation]
Revised: [Equation]

Original: where 0 pq means the p × q zero matrix.
Revised: where 0 pq means the p × q zero matrix.

Original: If l ′ i − l i = 0 and l′ i − 1 − l i − 1 > 0 , we set
Revised: If l ′ i − l i = 0 and l

Original: [Equation]
Revised: [Equation]

Original: If l ′ i − 1 − l i − 1= 0 and l ′ i − l i > 0 , we set 
Revised: i − 1 − l i − 1 > 0 , we set [Equation] b [Equation]

Revised: b

Original: [Equation]
Revised: [Equation]

Original: If l ′ i − 1 − l i − 1 = 0 and l ′ i − l i = 0 , we set
Revised: If l ′ i − 1 − l i − 1 = 0 and l ′ i − l i > 0 , we set

Original: [Equation]
Revised: [Equation]

Revised: If l ′ i − 1 − l i − 1 = 0 and l ′ i − l i = 0 , we set w ′ i =

Revised: w i , b ′ i =

Revised: b i .

Original: It is easy to check that if l ′ i − l i >
Revised: It is easy to check that if l ′ i − l i >

Original: [Equation]
Revised: [Equation]

Original: If l ′ i − l i = 0 , f ′ i
Revised: If l ′ i − l i = 0 , f ′ i

Original: = f i .
Revised: = f i .

Original: Since l ′ g − l g = 0 , f ′ g = f g , i.e. , f w ′ , b ′ =
Revised: Since l ′ g − l g = 0 , f ′ g = f g , i.e. , f w ′ , b ′ =

Original: f w , b .
Revised: f w , b .

Original: Therefore, f w , b ∈ F σ q ′ , which implies that F σ q ⊂ F σ q ′ .
Revised: Therefore, f w , b ∈ F σ q ′ , which implies that F σ q ⊂ F σ q ′ .

Original: Therefore, H σ q ⊂ H σ q ′ .
Revised: Therefore, H σ q ⊂ H σ q ′ .

Original: Lemma 10.
Revised: Lemma 10.

Original: Let σ be the ReLU function: σ ( x ) = max { x, 0 } .
Revised: Let σ be the ReLU function: σ ( x ) = max { x, 0 } .

Original: Then, q ≲ q ′ implies that F σ q ⊂ F σ q ′ ,
Revised: Then, q ≲ q ′ implies that F σ q ⊂ F σ q ′ ,

Original: H σ q ⊂ H σ q ′ , where q = ( l 1 , ..., l g ) and q ′ =
Revised: H σ q ⊂ H σ q ′ , where q = ( l 1 , ..., l g ) and q ′ =

Original: ( l ′ 1 , ..., l ′ g ′ ) .
Revised: ( l ′ 1 , ..., l ′ g ′ ) .

Original: Proof of Lemma 10.
Revised: Proof of Lemma 10.

Original: Given l ′′ =
Revised: Given l ′′ =

Original: ( l ′′ 1 , ..., l ′′ g ′′ ) satisfying that g ≤ g ′′ , l ′′
Revised: ( l ′′ 1 , ..., l ′′ g ′′ ) satisfying that g ≤ g ′′ , l ′′

Original: i =
Revised: i =

Original: l i for i = 1 , ..., g − 1 , l ′′
Revised: l i for i = 1 , ..., g − 1 , l ′′

Original: i =
Revised: i =

Original: l g − 1 for i = g, ..., g ′′ − 1 , and l ′′ g ′′ =
Revised: l g − 1 for i = g, ..., g ′′ − 1 , and l ′′ g ′′ =

Original: l g , we first prove that F σ q ⊂ F σ q ′′ and H σ q ⊂ H σ q ′′ .
Revised: l g , we first prove that F σ q ⊂ F σ q ′′ and H σ q ⊂ H σ q ′′ .

Original: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the i -th layer output of FCNN with architecture q can be written as
Revised: Given any weights w i ∈ R l i × l i − 1 and bias b i ∈ R l i × 1 , the i -th layer output of FCNN with architecture q can be written as

Original: [Equation]
Revised: [Equation]

Original: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .
Revised: where f i − 1 ( x ) is the i -th layer output and f 1 ( x ) = x .

Original: Then, the output of the last layer is
Revised: Then, the output of the last layer is

Original: [Equation]
Revised: [Equation]

Revised: g .

Original: We will show that f w , b ∈ F σ q ′′ .
Revised: We will show that f w , b ∈ F σ q ′′ .

Original: We construct f w ′′ , b ′′ as follows: if i = 2 , ..., g − 1 , then w ′′ i = w and b ′′ i =
Revised: We construct f w ′′ , b ′′ as follows: if i = 2 , ..., g − 1 , then w ′′ i = w and b ′′ i =

Original: b i ; if i = g, ..., g ′′ − 1 , then w ′′
Revised: b i ; if i = g, ..., g ′′ − 1 , then w ′′

Original: i =
Revised: i =

Original: I l g − 1 × l g [Equation] 1 and b ′′ i = 

Revised: I l g − 1 × l g − 1 and b ′′ i =

Original: l g − 1 × 1 , where I l g − 1 × l g − 1 is the l g − 1 × l g − 1 identity matrix, and 0 l g − 1 × 1 is the l g − 1 × 1 zero matrix; and if i = g ′′ , then w ′′ g ′′ = w g , b ′′ g ′′ =
Revised: l g − 1 × 1 , where I l g − 1 × l g − 1 is the l g − 1 × l g − 1 identity matrix, and 0 l g − 1 × 1 is the l g − 1 × 1 zero matrix; and if i = g ′′ , then w ′′ g ′′ = w g , b ′′ g ′′ =

Original: b g .
Revised: b g .

Original: Then it is easy to check that the output of the i -th layer is f ′′ i =
Revised: Then it is easy to check that the output of the i -th layer is f [Equation] i =

Original: f g − 1 , ∀ i = g − 1 , g, ..., g ′′ − 1 .
Revised: f g − 1 , ∀ i = g − 1 , g, ..., g ′′ − 1 .

Original: Therefore, f w ′′ , b ′′ = f w , b , which implies that F σ q ⊂ F σ q ′′ .
Revised: Therefore, f w ′′ , b ′′ = f w , b , which implies that F σ q ⊂ F σ q ′′ .

Original: Hence, H σ q ⊂ H σ q ′′ .
Revised: Hence, H σ q ⊂ H σ q ′′ .

Original: When g ′′ = g ′ , we use Lemma 9 ( q ′′ and q satisfy the condition in Lemma 9), which implies that
Revised: When g ′′ = g ′ , we use Lemma 9 ( q ′′ and q satisfy the condition in Lemma 9), which implies that

Original: F σ q ′′ ⊂
Revised: F σ q ′′ ⊂

Original: F σ q ′ , H σ q ′′ ⊂ H σ q ′ .
Revised: F σ q ′ , H σ q ′′ ⊂ H σ q ′ .

Original: Therefore, F σ q ⊂ F σ q ′ , H σ q ⊂ H σ q ′ .
Revised: Therefore, F σ q ⊂ F σ q ′ , H σ q ⊂ H σ q ′ .

Original: Lemma 11.
Revised: Lemma 11.

Original:  If the activation function σ is not a polynomial, then for any continuous function f defined in R d , and any compact set C ⊂ R d , there exists a fully-connected neural network with architecture q ( l 1 = d, l g = 1) such that
Revised: [85] If the activation function σ is not a polynomial, then for any continuous function f defined in R d , and any compact set C ⊂ R d , there exists a fully-connected neural network with architecture q ( l 1 = d, l g = 1) such that

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma 11.
Revised: Proof of Lemma 11.

Original: The proof of Lemma 11 can be found in Theorem 3.1 in [79].
Revised: The proof of Lemma 11 can be found in Theorem 3.1 in [85].

Original: Lemma 12.
Revised: Lemma 12.

Original: If the activation function σ is the ReLU function, then for any continuous vector-valued function f ∈ C ( R d ; R l ) , and any compact set C ⊂ R d , there exists a fully-connected neural network with architecture q ( l 1 = d, l g = l ) such that
Revised: If the activation function σ is the ReLU function, then for any continuous vector-valued function f ∈ C ( R d ; R l ) , and any compact set C ⊂ R d , there exists a fully-connected neural network with architecture q ( l 1 = d, l g = l ) such that

Original: [Equation]
Revised: [Equation]

Original: where ∥ · ∥ 2 is the ℓ 2 norm.
Revised: where ∥ · ∥ 2 is the ℓ 2 norm.

Original: ( Note that we can also prove the same result, if σ is not a polynomial. )
Revised: ( Note that we can also prove the same result, if σ is not a polynomial. )

Original: Proof of Lemma 12.
Revised: Proof of Lemma 12.

Original: Let f =
Revised: Let f =

Original: [ f 1 , ..., f l ] ⊤ , where f i is the i -th coordinate of f .
Revised: [ f 1 , ..., f l ] ⊤ , where f i is the i -th coordinate of f .

Original: Based on Lemma 11, we obtain l sequences q 1 , q 2 ,..., q l such that
Revised: Based on Lemma 11, we obtain l sequences q 1 , q 2 ,..., q l such that

Original: [Equation]
Revised: [Equation]

Original: It is easy to find a sequence q = ( l 1 , ..., l g )
Revised: It is easy to find a sequence q = ( l 1 , ..., l g )

Original: ( l g = 1 ) such that q i ≲ q , for all i = 1 , ..., l .
Revised: ( l g = 1 ) such that q i ≲ q , for all i = 1 , ..., l .

Original: Using
Revised: Using

Original: Lemma 10, we obtain that F σ q i ⊂ F σ q .
Revised: Lemma 10, we obtain that F σ q i ⊂ F σ q .

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: Therefore, for each i , we can find g w i , b i from F σ q such that
Revised: Therefore, for each i , we can find g w i , b i from F σ q such that

Original: [Equation]
Revised: [Equation]

Original: where w i represents weights and b i represents bias.
Revised: where w i represents weights and b i represents bias.

Original: We construct a larger FCNN with q ′ =
Revised: We construct a larger FCNN with q ′ =

Original: ( l ′ 1 , l ′ 2 , ..., l ′ g ) satisfying that l ′ 1 =
Revised: ( l ′ 1 , l ′ 2 , ..., l ′ g ) satisfying that l ′ 1 =

Original: d , l ′ i
Revised: d , l ′ i

Original: = l ∗ l i , for i = 2 , ..., g .
Revised: = l ∗ l i , for i = 2 , ..., g .

Original: We can regard this larger FCNN as a combinations of l FCNNs with architecture q , that is: there are m disjoint sub-FCNNs with architecture q in the larger FCNN with architecture q ′ .
Revised: We can regard this larger FCNN as a combinations of l FCNNs with architecture q , that is: there are m disjoint sub-FCNNs with architecture q in the larger FCNN with architecture q ′ .

Original: For i -th sub-FCNN, we use weights w i and bias b i .
Revised: For i -th sub-FCNN, we use weights w i and bias b i .

Original: For weights and bias which connect different sub-FCNNs, we set these weights and bias to 0 .
Revised: For weights and bias which connect different sub-FCNNs, we set these weights and bias to 0 .

Original: Finally, we can obtain that g w , b =
Revised: Finally, we can obtain that g w , b =

Original: [ g w 1 , b 1 , g w 2 , b 2 , ..., g w l , b l ] ⊤ ∈ F σ q ′ , which implies that
Revised: [ g w 1 , b 1 , g w 2 , b 2 , ..., g w l , b l ] ⊤ ∈ F σ q ′ , which implies that

Original: [Equation]
Revised: [Equation]

Original: We have completed this proof.
Revised: We have completed this proof.

Original: Given a sequence q = ( l 1 , ..., l g ) , we are interested in following function space F σ q , M :
Revised: Given a sequence q = ( l 1 , ..., l g ) , we are interested in following function space F σ q , M :

Original: [Equation]
Revised: [Equation]

Original: where ◦ means the composition of two functions, · means the product of two matrices, and
Revised: where ◦ means the composition of two functions, · means the product of two matrices, and

Original: [Equation]
Revised: [Equation]

Original: here 1 1 × ( l g − 1) is the 1 × ( l g − 1) matrix whose all elements are 1 , and 0 1 × ( l g − 1) is the 1 × ( l g − 1) zero matrix.
Revised: here 1 1 × ( l g − 1) is the 1 × ( l g − 1) matrix whose all elements are 1 , and 0 1 × ( l g − 1) is the 1 × ( l g − 1) zero matrix.

Original: Using F σ q , M , we can construct a binary classification space H σ q , M , which consists of all classifiers satisfying the following condition:
Revised: Using F σ q , M , we can construct a binary classification space H σ q , M , which consists of all classifiers satisfying the following condition:

Original: [Equation]
Revised: [Equation]

Original: where f k M ( x ) is the k -th coordinate of M · ( σ ◦ f ) .
Revised: where f k M ( x ) is the k -th coordinate of M · ( σ ◦ f ) .

Original: Lemma 13.
Revised: Lemma 13.

Original: Suppose that σ is the ReLU function: max { x, 0 } .
Revised: Suppose that σ is the ReLU function: max { x, 0 } .

Original: Given a sequence q
Revised: Given a sequence q

Original: =
Revised: =

Original: ( l 1 , ..., l g ) satisfying that l 1 = d and l g = K + 1 , then the space H σ q , M contains ϕ ◦
Revised: ( l 1 , ..., l g ) satisfying that l 1 = d and l g = K + 1 , then the space H σ q , M contains ϕ ◦

Original: H σ q , and H σ q , M has finite
Revised: H σ q , and H σ q , M has finite

Original: VC dimension ( Vapnik–Chervonenkis dimension ) , where ϕ maps ID data to 1 and OOD data to 2 .
Revised: VC dimension ( Vapnik–Chervonenkis dimension ) , where ϕ maps ID data to 1 and OOD data to 2 .

Original: Furthermore, if given q ′ =
Revised: Furthermore, if given q ′ =

Original: ( l ′ 1 , ..., l ′ g ) satisfying that l ′ g
Revised: ( l ′ 1 , ..., l ′ g ) satisfying that l ′ g

Original: = K and l ′
Revised: = K and l ′

Original: i
Revised: i

Original: =
Revised: =

Original: l i , for i = 1 , ..., g − 1 , then
Revised: l i , for i = 1 , ..., g − 1 , then

Original: H σ q ⊂ H σ q ′ • H σ q , M .
Revised: H σ q ⊂ H σ q ′ • H σ q , M .

Original: Proof of Lemma 13.
Revised: Proof of Lemma 13.

Original: For any h w , b ∈ H σ q , then there exists f w , b ∈ F σ q such that h w , b is induced by f w , b .
Revised: For any h w , b ∈ H σ q , then there exists f w , b ∈ F σ q such that h w , b is induced by f w , b .

Original: We can write f w , b as follows:
Revised: We can write f w , b as follows:

Original: [Equation]
Revised: [Equation]

Original: where w g ∈ R ( K +1)
Revised: where w g ∈ R ( K +1)

Original: × l g − 1 , b g ∈ R ( K +1) × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.
Revised: × l g − 1 , b g ∈ R ( K +1) × 1 and f g − 1 ( x ) is the output of the l g − 1 -th layer.

Original: Suppose that
Revised: Suppose that

Original: [Equation]
Revised: [Equation]

Original: 
Revised: 

Original: [Equation]
Revised: [Equation]

Original: where v i ∈ R 1 × l g − 1 and b i ∈ R .
Revised: where v i ∈ R 1 × l g − 1 and b i ∈ R .

Original: We set
Revised: We set

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: It is obvious that f w ′ , b ′ ∈ F σ q ′ .
Revised: It is obvious that f w ′ , b ′ ∈ F σ q ′ .

Original: Using f w ′ , b ′ ∈ F σ q ′ , we construct a classifier h w ′ , b ′ ∈ H σ q ′ :
Revised: Using f w ′ , b ′ ∈ F σ q ′ , we construct a classifier h w ′ , b ′ ∈ H σ q ′ :

Original: [Equation]
Revised: [Equation]

Original: where f k w ′ , b ′ is the k -th coordinate of f w ′ , b ′ .
Revised: where f k w ′ , b ′ is the k -th coordinate of f w ′ , b ′ .

Original: Additionally, we consider
Revised: Additionally, we consider

Original: [Equation]
Revised: [Equation]

Original: where
Revised: where

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: here I ( l g − 1) ×
Revised: here I ( l g − 1) ×

Original: ( l g − 1) is the ( l g − 1) ×
Revised: ( l g − 1) is the ( l g − 1) ×

Original: ( l g − 1) identity matrix, 0 1 × ( l g − 1) is the 1 × ( l g − 1) zero matrix, and 1 ( l g − 1) × 1 is the ( l g − 1) × 1 matrix, whose all elements are 1 .
Revised: ( l g − 1) identity matrix, 0 1 × ( l g − 1) is the 1 × ( l g − 1) zero matrix, and 1 ( l g − 1) × 1 is the ( l g − 1) × 1 matrix, whose all elements are 1 .

Original: Then, we define that for any x ∈ X ,
Revised: Then, we define that for any x ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: where f k w , b , B ( x ) is the k -th coordinate of f w , b , B ( x ) .
Revised: where f k w , b , B ( x ) is the k -th coordinate of f w , b , B ( x ) .

Original: Furthermore, we can check that h w , b , B can be written as follows: for any x ∈ X ,
Revised: Furthermore, we can check that h w , b , B can be written as follows: for any x ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: w
Revised: w

Original: [Equation]
Revised: [Equation]

Original: b
Revised: b

Original: [Equation]
Revised: [Equation]

Original: It is easy to check that
Revised: It is easy to check that

Original: [Equation]
Revised: [Equation]

Original: where ϕ maps ID labels to 1 and OOD labels to 2 .
Revised: where ϕ maps ID labels to 1 and OOD labels to 2 .

Original: Therefore, h w , b ( x ) = K + 1 if
Revised: Therefore, h w , b ( x ) = K + 1 if

Original: and only if h w , b , B = 2 ; and h w , b ( x ) =
Revised: and only if h w , b , B = 2 ; and h w , b ( x ) =

Original: k ( k ̸ = K + 1 )
Revised: k ( k ̸ = K + 1 )

Original: if and only if h w , b , B = 1 and h w ′ , b ′ ( x ) = k .
Revised: if and only if h w , b , B = 1 and h w ′ , b ′ ( x ) = k .

Original: This implies that H σ q ⊂ H σ q ′ • H σ q , M and ϕ ◦ H σ q ⊂ H σ q , M .
Revised: This implies that H σ q ⊂ H σ q ′ • H σ q , M and ϕ ◦ H σ q ⊂ H σ q , M .

Original: Let ˜ q be ( l 1 , ..., l g , 2) .
Revised: Let ˜ q be ( l 1 , ..., l g , 2) .

Original: Then F σ q , M ⊂ F σ ˜ q .
Revised: Then F σ q , M ⊂ F σ ˜ q .

Original: Hence, H σ q , M ⊂ H σ ˜ q .
Revised: Hence, H σ q , M ⊂ H σ ˜ q .

Original: According to the VC dimension theory [37] for feed-forward neural networks, H σ ˜ q has finite VC dimension.
Revised: According to the VC dimension theory [37] for feed-forward neural networks, H σ ˜ q has finite VC dimension.

Original: Hence, H σ q , M has finite
Revised: Hence, H σ q , M has finite

Original: VC-dimension.
Revised: VC-dimension.

Original: We have completed the proof.
Revised: We have completed the proof.

Original: Lemma 14.
Revised: Lemma 14.

Original: Let |X| < + ∞ and σ be the ReLU function: max { x, 0 } .
Revised: Let |X| < + ∞ and σ be the ReLU function: max { x, 0 } .

Original: Given r hypothesis functions h 1 , h 2 , ..., h r ∈ { h : X → { 1 , ..., l }} , then there exists a sequence q =
Revised: Given r hypothesis functions h 1 , h 2 , ..., h r ∈ { h : X → { 1 , ..., l }} , then there exists a sequence q =

Original: ( l 1 , ..., l g ) with l 1 = d and l g = l , such that h 1 , ..., h r ∈ H σ q .
Revised: ( l 1 , ..., l g ) with l 1 = d and l g = l , such that h 1 , ..., h r ∈ H σ q .

Original: Proof of Lemma 14.
Revised: Proof of Lemma 14.

Original: For each h i ( i = 1 , ..., r ), we introduce a corresponding f i (defined over X ) satisfying that for any x ∈ X , f i ( x ) =
Revised: For each h i ( i = 1 , ..., r ), we introduce a corresponding f i (defined over X ) satisfying that for any x ∈ X , f i ( x ) =

Original: y k
Revised: y k

Original: if and only if h i ( x ) = k , where y k ∈ R l is the one-hot vector corresponding to the label k .
Revised: if and only if h i ( x ) = k , where y k ∈ R l is the one-hot vector corresponding to the label k .

Original: Clearly, f i is a continuous function in X , because X is a discrete set.
Revised: Clearly, f i is a continuous function in X , because X is a discrete set.

Original: Tietze Extension Theorem implies that f i can be extended to a continuous function in R d .
Revised: Tietze Extension Theorem implies that f i can be extended to a continuous function in R d .

Original: Since X is a compact set, then Lemma 12 implies that there exist a sequence q i =
Revised: Since X is a compact set, then Lemma 12 implies that there exist a sequence q i =

Original: ( l i 1 , ..., l ig i ) ( l i 1 = d and l ig i
Revised: ( l i 1 , ..., l ig i ) ( l i 1 = d and l ig i

Original: = l ) and f w , b ∈ F σ q i such that
Revised: = l ) and f w , b ∈ F σ q i such that

Original: [Equation]
Revised: [Equation]

Original: where ∥ · ∥ ℓ 2 is the ℓ 2 norm in R l .
Revised: where ∥ · ∥ ℓ 2 is the ℓ 2 norm in R l .

Original: Therefore, for any x ∈ X , it easy to check that
Revised: Therefore, for any x ∈ X , it easy to check that

Original: [Equation]
Revised: [Equation]

Original: where f k w , b ( x ) is the k -th coordinate of f w , b ( x ) .
Revised: where f k w , b ( x ) is the k -th coordinate of f w , b ( x ) .

Original: Therefore, h i ( x ) ∈ H σ q i .
Revised: Therefore, h i ( x ) ∈ H σ q i .

Original: Let q be ( l 1 , ..., l g ) ( l 1 = d and l g = l ) satisfying that q i ≲ q .
Revised: Let q be ( l 1 , ..., l g ) ( l 1 = d and l g = l ) satisfying that q i ≲ q .

Original: Using Lemma 10, we obtain that
Revised: Using Lemma 10, we obtain that

Original: H σ q i ⊂ H σ q , for each i = 1 , ..., r .
Revised: H σ q i ⊂ H σ q , for each i = 1 , ..., r .

Original: Therefore, h 1 , ..., h r ∈ H σ q .
Revised: Therefore, h 1 , ..., h r ∈ H σ q .

Original: Lemma 15.
Revised: Lemma 15.

Original: Let the activation function σ be the ReLU function.
Revised: Let the activation function σ be the ReLU function.

Original: Suppose that |X| < + ∞ .
Revised: Suppose that |X| < + ∞ .

Original: If { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets of R l ( here, open set is a topological terminology ) .
Revised: If { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets of R l ( here, open set is a topological terminology ) .

Original: There exists a sequence q =
Revised: There exists a sequence q =

Original: ( l 1 , ..., l g )
Revised: ( l 1 , ..., l g )

Original: ( l 1 = d and l g = l ) such that H σ,λ q ,E consists of all binary classifiers.
Revised: ( l 1 = d and l g = l ) such that H σ,λ q ,E consists of all binary classifiers.

Original: Proof of Lemma 15.
Revised: Proof of Lemma 15.

Original: Since { v ∈ R l : E ( v ) ≥ λ } , { v ∈ R l : E ( v ) < λ } both contain nonempty open sets, we can find v 1 ∈ { v ∈ R l : E ( v ) ≥ λ } , v 2 ∈ { v ∈ R l : E ( v ) < λ } and a constant r > 0 such that B r ( v 1 ) ⊂ { v ∈ R l : E ( v ) ≥ λ } and B r ( v 2 ) ⊂ { v ∈ R l : E ( v ) < λ } , where
Revised: Since { v ∈ R l : E ( v ) ≥ λ } , { v ∈ R l : E ( v ) < λ } both contain nonempty open sets, we can find v 1 ∈ { v ∈ R l : E ( v ) ≥ λ } , v 2 ∈ { v ∈ R l : E ( v ) < λ } and a constant r > 0 such that B r ( v 1 ) ⊂ { v ∈ R l : E ( v ) ≥ λ } and B r ( v 2 ) ⊂ { v ∈ R l : E ( v ) < λ } , where

Original: B r ( v 1 )
Revised: B r ( v 1 )

Original: = { v : ∥ v − v 1 ∥ ℓ 2 < r } and B r ( v 2 ) =
Revised: = { v : ∥ v − v 1 ∥ ℓ 2 < r } and B r ( v 2 ) =

Original: { v : ∥ v − v 2 ∥ ℓ 2 < r } , here ∥ · ∥ ℓ 2 is the ℓ 2 norm.
Revised: { v : ∥ v − v 2 ∥ ℓ 2 < r } , here ∥ · ∥ ℓ 2 is the ℓ 2 norm.

Original: For any binary classifier h over X , we can induce a vector-valued function as follows: for any x ∈ X , f ( x )
Revised: For any binary classifier h over X , we can induce a vector-valued function as follows: for any x ∈ X , f ( x )

Original: [Equation]
Revised: [Equation]

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: Since X is a finite set, then Tietze Extension Theorem implies that f can be extended to a continuous function in R d .
Revised: Since X is a finite set, then Tietze Extension Theorem implies that f can be extended to a continuous function in R d .

Original: Since X is a compact set, Lemma 12 implies that there exists a sequence q h
Revised: Since X is a compact set, Lemma 12 implies that there exists a sequence q h

Original: = ( l h 1 , ..., l hg h ) ( l h 1 = d and l hg h = l ) and f w , b ∈ F σ q h such that
Revised: = ( l h 1 , ..., l hg h ) ( l h 1 = d and l hg h = l ) and f w , b ∈ F σ q h such that

Original: [Equation]
Revised: [Equation]

Original: where ∥ · ∥ ℓ 2 is the ℓ 2 norm in R l .
Revised: where ∥ · ∥ ℓ 2 is the ℓ 2 norm in R l .

Original: Therefore, for any x ∈ X , it is easy to check that E ( f w , b ( x )) ≥ λ
Revised: Therefore, for any x ∈ X , it is easy to check that E ( f w , b ( x )) ≥ λ

Original: if and only if h ( x ) = 1 , and E ( f w , b ( x )) <
Revised: if and only if h ( x ) = 1 , and E ( f w , b ( x )) <

Original: λ if and only if h ( x ) = 2 .
Revised: λ if and only if h ( x ) = 2 .

Original: For each h , we have found a sequence q h such that h is induced by f w , b ∈ F σ q h , E and λ .
Revised: For each h , we have found a sequence q h such that h is induced by f w , b ∈ F σ q h , E and λ .

Original: Since |X| < + ∞ , only finite binary classifiers are defined over X .
Revised: Since |X| < + ∞ , only finite binary classifiers are defined over X .

Original: Using Lemma 14, we can find a sequence q such that H b all =
Revised: Using Lemma 14, we can find a sequence q such that H b all =

Original: H σ,λ q ,E , where H b all consists of all binary classifiers.
Revised: H σ,λ q ,E , where H b all consists of all binary classifiers.

Original: Lemma 16.
Revised: Lemma 16.

Original: Suppose the hypothesis space is score-based.
Revised: Suppose the hypothesis space is score-based.

Original: Let |X| < + ∞ .
Revised: Let |X| < + ∞ .

Original: If { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets, and Condition 2 holds, then there exists a sequence q = ( l 1 , ..., l g )
Revised: If { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets, and Condition 2 holds, then there exists a sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = l ) such that for any sequence q ′ satisfying q ≲ q ′ and any ID hypothesis space H in , OOD detection is learnable in the separate space D sXY for H in • H b , where H b = H σ,λ q ′ ,E and H in • H b is defined below Eq.
Revised: ( l 1 = d and l g = l ) such that for any sequence q ′ satisfying q ≲ q ′ and any ID hypothesis space H in , OOD detection is learnable in the separate space D sXY for H in • H b , where H b = H σ,λ q ′ ,E and H in • H b is defined below Eq.

Original: Proof of Lemma 16.
Revised: Proof of Lemma 16.

Original: Note that we use the ReLU function as the activation function in this lemma.
Revised: Note that we use the ReLU function as the activation function in this lemma.

Original: Using Lemma 10, Lemma 15 and Theorem 7, we can prove this result.
Revised: Using Lemma 10, Lemma 15 and Theorem 7, we can prove this result.

Original: Theorem 10.
Revised: Theorem 10.

Original: Suppose that Condition 2 holds and the hypothesis space H is FCNN-based or scorebased
Revised: Suppose that Condition 2 holds and the hypothesis space H is FCNN-based or scorebased

Original: , i.e., H = H σ q or H =
Revised: , i.e., H = H σ q or H =

Original: H in • H b , where H in is an ID hypothesis space, H b
Revised: H in • H b , where H in is an ID hypothesis space, H b

Original: = H σ,λ q ,E and
Revised: = H σ,λ q ,E and

Original: H = H in • H b is introduced below Eq.
Revised: H = H in • H b is introduced below Eq.

Original: (4) , here E is introduced in Eqs.
Revised: (4) , here E is introduced in Eqs.

Original: (5) or (6) .
Revised: (5) or (6) .

Original: Then
Revised: Then

Original: There is a sequence q =
Revised: There is a sequence q =

Original: ( l 1 , ..., l g )
Revised: ( l 1 , ..., l g )

Original: such that OOD detection is learnable in the separate space D sXY for H if and only if |X| < + ∞ .
Revised: such that OOD detection is learnable in the separate space D sXY for H if and only if |X| < + ∞ .

Original: Furthermore, if |X| < + ∞ , then there exists a sequence q =
Revised: Furthermore, if |X| < + ∞ , then there exists a sequence q =

Original: ( l 1 , ..., l g )
Revised: ( l 1 , ..., l g )

Original: such that for any sequence q ′ satisfying that q ≲ q ′ , OOD detection is learnable in D sXY for H .
Revised: such that for any sequence q ′ satisfying that q ≲ q ′ , OOD detection is learnable in D sXY for H .

Original: Proof of Theorem 10.
Revised: Proof of Theorem 10.

Original: Note that we use the ReLU function as the activation function in this theorem.
Revised: Note that we use the ReLU function as the activation function in this theorem.

Original: • The Case that H is FCNN-based.
Revised: • The Case that H is FCNN-based.

Original: First , we prove that if |X| = + ∞ , then OOD detection is not learnable in D sXY for H σ q , for any sequence q = ( l 1 , ..., l g )
Revised: First , we prove that if |X| = + ∞ , then OOD detection is not learnable in D sXY for H σ q , for any sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = K + 1) .
Revised: ( l 1 = d and l g = K + 1) .

Original: By Lemma 13, Theorems 5 and 8 in [80], we know that VCdim( ϕ ◦ H σ q ) < + ∞ , where ϕ maps ID data to 1 and maps OOD data to 2 .
Revised: By Lemma 13, Theorems 5 and 8 in [86], we know that VCdim( ϕ ◦ H σ q ) < + ∞ , where ϕ maps ID data to 1 and maps OOD data to 2 .

Original: Additionally, Proposition 1 implies that Assumption 1 holds and sup h ∈H σ q |{ x ∈ X : h ( x ) ∈ Y}| =
Revised: Additionally, Proposition 1 implies that Assumption 1 holds and sup h ∈H σ q |{ x ∈ X : h ( x ) ∈ Y}| =

Original: + ∞ , when |X| =
Revised: + ∞ , when |X| =

Original: + ∞ .
Revised: + ∞ .

Original: Therefore, Theorem 5 implies that OOD detection is not learnable in D sXY for H σ q , when |X| = + ∞ .
Revised: Therefore, Theorem 5 implies that OOD detection is not learnable in D sXY for H σ q , when |X| = + ∞ .

Original: Second , we prove that if |X| < + ∞ , there exists a sequence q = ( l 1 , ..., l g )
Revised: Second , we prove that if |X| < + ∞ , there exists a sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = K +1)
Revised: ( l 1 = d and l g = K +1)

Original: such that OOD detection is learnable in D sXY for H σ q .
Revised: such that OOD detection is learnable in D sXY for H σ q .

Original: Since |X| < + ∞ , it is clear that |H all | < + ∞ , where H all consists of all hypothesis functions from
Revised: Since |X| < + ∞ , it is clear that |H all | < + ∞ , where H all consists of all hypothesis functions from

Original: X to Y all .
Revised: X to Y all .

Original: According to Lemma 14, there exists a sequence q such that H all ⊂ H σ q .
Revised: According to Lemma 14, there exists a sequence q such that H all ⊂ H σ q .

Original: Additionally,
Revised: Additionally,

Original: Lemma 13 implies that there exist H in and H b such that H σ q ⊂ H in • H b .
Revised: Lemma 13 implies that there exist H in and H b such that H σ q ⊂ H in • H b .

Original: Since H all consists all hypothesis space, H all = H σ q = H in • H b .
Revised: Since H all consists all hypothesis space, H all = H σ q = H in • H b .

Original: Therefore, H b contains all binary classifiers from X to { 1 , 2 } .
Revised: Therefore, H b contains all binary classifiers from X to { 1 , 2 } .

Original: Theorem 7 implies that OOD detection is learnable in D sXY for H σ q .
Revised: Theorem 7 implies that OOD detection is learnable in D sXY for H σ q .

Original: Third , we prove that if |X| < + ∞ , then there exists a sequence q = ( l 1 , ..., l g )
Revised: Third , we prove that if |X| < + ∞ , then there exists a sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = K + 1) such that for any sequence q ′ =
Revised: ( l 1 = d and l g = K + 1) such that for any sequence q ′ =

Original: ( l ′ 1 , ..., l ′ g ′ )
Revised: ( l ′ 1 , ..., l ′ g ′ )

Original: satisfying that q ≲ q ′ , OOD detection is learnable in D sXY for H σ q ′ .
Revised: satisfying that q ≲ q ′ , OOD detection is learnable in D sXY for H σ q ′ .

Original: We can use the sequence q constructed in the second step of the proof.
Revised: We can use the sequence q constructed in the second step of the proof.

Original: Therefore, H σ q = H all .
Revised: Therefore, H σ q = H all .

Original: Lemma 10 implies that H σ q ⊂ H σ q ′ .
Revised: Lemma 10 implies that H σ q ⊂ H σ q ′ .

Original: Therefore, H σ q ′ =
Revised: Therefore, H σ q ′ =

Original: H all = H σ q .
Revised: H all = H σ q .

Original: The proving process (second step of the proof) has shown that if |X| < + ∞ , Condition 2 holds and hypothesis space H consists of all hypothesis functions, then OOD detection is learnable in D sXY for H .
Revised: The proving process (second step of the proof) has shown that if |X| < + ∞ , Condition 2 holds and hypothesis space H consists of all hypothesis functions, then OOD detection is learnable in D sXY for H .

Original: Therefore, OOD detection is learnable in D sXY for H σ q ′ .
Revised: Therefore, OOD detection is learnable in D sXY for H σ q ′ .

Original: We complete the proof when the hypothesis space H is FCNN-based.
Revised: We complete the proof when the hypothesis space H is FCNN-based.

Original: The
Revised: The

Original: Case that H is score-based
Revised: Case that H is score-based

Original: Fourth , we prove that if |X| = + ∞ , then OOD detection is not learnable in D sXY for H in • H b , where
Revised: Fourth , we prove that if |X| = + ∞ , then OOD detection is not learnable in D sXY for H in • H b , where

Original: H b = H σ,λ q ,E for any sequence q = ( l 1 , ..., l g ) ( l 1 = d , l g = l ), where E is in Eqs.
Revised: H b = H σ,λ q ,E for any sequence q = ( l 1 , ..., l g ) ( l 1 = d , l g = l ), where E is in Eqs.

Original: (5) or (6).
Revised: (5) or (6).

Original: By Theorems 5 and 8 in [80], we know that VCdim( H σ,λ q ,E ) <
Revised: By Theorems 5 and 8 in [86], we know that VCdim( H σ,λ q ,E ) <

Original: + ∞ .
Revised: + ∞ .

Original: Additionally, Propositionimplies that Assumption 1 holds and sup h ∈H σ q |{ x ∈ X : h ( x )
Revised: Additionally, Propositionimplies that Assumption 1 holds and sup h ∈H σ q |{ x ∈ X : h ( x )

Original: ∈ Y}| =
Revised: ∈ Y}| =

Original: + ∞ , when |X| =
Revised: + ∞ , when |X| =

Original: + ∞ .
Revised: + ∞ .

Original: Hence, Theorem 5 implies that OOD detection is not learnable in D sXY for H σ q , when |X| = + ∞ .
Revised: Hence, Theorem 5 implies that OOD detection is not learnable in D sXY for H σ q , when |X| = + ∞ .

Original: Fifth , we prove that if |X| < + ∞ , there exists a sequence q = ( l 1 , ..., l g )
Revised: Fifth , we prove that if |X| < + ∞ , there exists a sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = l ) such that OOD detection is learnable in D sXY for for H in • H b , where H b = H σ,λ q ,E for any sequence q = ( l 1 , ..., l g ) ( l 1 = d , l g = l ), where E is in Eq.
Revised: ( l 1 = d and l g = l ) such that OOD detection is learnable in D sXY for for H in • H b , where H b = H σ,λ q ,E for any sequence q = ( l 1 , ..., l g ) ( l 1 = d , l g = l ), where E is in Eq.

Original: (5) or Eq.
Revised: (5) or Eq.

Original: Based on Lemma 16, we only need to show that { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets for different scoring functions E .
Revised: Based on Lemma 16, we only need to show that { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) < λ } both contain nonempty open sets for different scoring functions E .

Original: Since max k ∈{ 1 ,...,l } exp( v k ) (cid:80) lc =1 exp( v c ) , max k ∈{ 1 ,...,l } exp( v k /T )
Revised: Since max k ∈{ 1 ,...,l } exp( v k ) (cid:80) lc =1 exp( v c ) , max k ∈{ 1 ,...,l } exp( v k /T )

Original: (cid:80)
Revised: (cid:80)

Original: Kc =1 exp( v c /T ) and T log (cid:80) lc
Revised: Kc =1 exp( v c /T ) and T log (cid:80) lc

Original: =1 exp ( v c /T ) are continuous functions, whose ranges contain ( 1 l , 1) , ( 1 l , 1) , (0 , + ∞ ) and (0 , + ∞ ) , respectively.
Revised: =1 exp ( v c /T ) are continuous functions, whose ranges contain ( 1 l , 1) , ( 1 l , 1) , (0 , + ∞ ) and (0 , + ∞ ) , respectively.

Original: Based on the property of continuous function ( E − 1 ( A ) is an open set, if A is an open set), we obtain that { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) <
Revised: Based on the property of continuous function ( E − 1 ( A ) is an open set, if A is an open set), we obtain that { v ∈ R l : E ( v ) ≥ λ } and { v ∈ R l : E ( v ) <

Original: λ } both contain nonempty open sets.
Revised: λ } both contain nonempty open sets.

Original: Using
Revised: Using

Original: Lemma 16, we complete the fifth step.
Revised: Lemma 16, we complete the fifth step.

Original: Sixth , we prove that if |X| < + ∞ , then there exists a sequence q = ( l 1 , ..., l g )
Revised: Sixth , we prove that if |X| < + ∞ , then there exists a sequence q = ( l 1 , ..., l g )

Original: ( l 1 = d and l g = l ) such that for any sequence q ′ =
Revised: ( l 1 = d and l g = l ) such that for any sequence q ′ =

Original: ( l ′ 1 , ..., l ′ g ′ )
Revised: ( l ′ 1 , ..., l ′ g ′ )

Original: satisfying that q ≲ q ′ , OOD detection is learnable in
Revised: satisfying that q ≲ q ′ , OOD detection is learnable in

Original: D sXY for for H in • H b , where H b = H σ,λ q ′ ,E , where E is in Eq.
Revised: D sXY for for H in • H b , where H b = H σ,λ q ′ ,E , where E is in Eq.

Original: (5) or Eq.
Revised: (5) or Eq.

Original: In the fifth step, we have proven that Eq.
Revised: In the fifth step, we have proven that Eq.

Original: (5) and Eq.
Revised: (5) and Eq.

Original: (6) meet the condition in Lemma 16.
Revised: (6) meet the condition in Lemma 16.

Original: Therefore,
Revised: Therefore,

Original: Lemma 16 implies this result.
Revised: Lemma 16 implies this result.

Original: We complete the proof when the hypothesis space H is score-based.
Revised: We complete the proof when the hypothesis space H is score-based.

Original: M Proofs of Theorem 11 and TheoremM.1 Proof of Theorem
Revised: M Proofs of Theorem 11 and TheoremM.1 Proof of Theorem

Original: Theorem 11.
Revised: Theorem 11.

Original: Suppose that each domain D XY in D µ,bXY is attainable, i.e., arg min
Revised: Suppose that each domain D XY in D µ,bXY is attainable, i.e., arg min

Original: h ∈H R D ( h ) ̸
Revised: h ∈H R D ( h ) ̸

Original: = ∅ (the finite discrete domains satisfy this).
Revised: = ∅ (the finite discrete domains satisfy this).

Original: Let K = 1 and the hypothesis space H be score-based (
Revised: Let K = 1 and the hypothesis space H be score-based (

Original: H = H σ,λ q ,E , where E is in Eqs.
Revised: H = H σ,λ q ,E , where E is in Eqs.

Original: (5) or (6) ) or FCNN-based ( H = H σ q ) .
Revised: (5) or (6) ) or FCNN-based ( H = H σ q ) .

Original: If µ ( X ) < + ∞ , then the following four conditions are equivalent :
Revised: If µ ( X ) < + ∞ , then the following four conditions are equivalent :

Original: Learnability in D µ,bXY for H ⇐⇒ Condition 1 ⇐⇒
Revised: Learnability in D µ,bXY for H ⇐⇒ Condition 1 ⇐⇒

Original: Realizability Assumption ⇐⇒ Condition
Revised: Realizability Assumption ⇐⇒ Condition

Original: Proof of Theorem 11.
Revised: Proof of Theorem 11.

Original: 1) By Lemma 1, we conclude that Learnability in D µ,bXY for H ⇒ Condition 1. 2)
Revised: 1) By Lemma 1, we conclude that Learnability in D µ,bXY for H ⇒ Condition 1. 2)

Original: By Proposition 1 and Proposition 2, we know that when K = 1 , there exist h 1 , h 2 ∈ H , where h 1 = 1 and h 2 = 2 , here 1 represents ID, and 2 represent OOD.
Revised: By Proposition 1 and Proposition 2, we know that when K = 1 , there exist h 1 , h 2 ∈ H , where h 1 = 1 and h 2 = 2 , here 1 represents ID, and 2 represent OOD.

Original: Therefore, we know that when K = 1 , inf h ∈H R in D ( h ) = 0 and inf h ∈H R out D ( h )
Revised: Therefore, we know that when K = 1 , inf h ∈H R in D ( h ) = 0 and inf h ∈H R out D ( h )

Original: = 0 , for any D XY ∈ D µ,bXY .
Revised: = 0 , for any D XY ∈ D µ,bXY .

Original: By Condition 1, we obtain that inf h ∈H R D  = 0 , for any D XY ∈ D µ,bXY .
Revised: By Condition 1, we obtain that inf h ∈H R D ( h ) = 0 , for any D XY ∈ D µ,bXY .

Original: Because each domain D XY in D µ,bXY is attainable, we conclude that Realizability Assumption holds.
Revised: Because each domain D XY in D µ,bXY is attainable, we conclude that Realizability Assumption holds.

Original: We have proven that Condition 1 ⇒ Realizability Assumption.
Revised: We have proven that Condition 1 ⇒ Realizability Assumption.

Original: By Theorems 5 and 8 in [80], we know that VCdim( ϕ ◦ H σ q ) <
Revised: By Theorems 5 and 8 in [86], we know that VCdim( ϕ ◦ H σ q ) <

Original: + ∞ and VCdim( H σ,λ q ,E ) <
Revised: + ∞ and VCdim( H σ,λ q ,E ) <

Original: + ∞ .
Revised: + ∞ .

Original: Then, using Theorem 9, we conclude that Realizability Assumption ⇒ Learnability in D µ,bXY for H .
Revised: Then, using Theorem 9, we conclude that Realizability Assumption ⇒ Learnability in D µ,bXY for H .

Original: According to the results in 1), 2) and 3), we have proven that
Revised: According to the results in 1), 2) and 3), we have proven that

Original: Learnability in D µ,bXY for H ⇔ Condition 1 ⇔ Realizability Assumption.
Revised: Learnability in D µ,bXY for H ⇔ Condition 1 ⇔ Realizability Assumption.

Original: 5) By Lemma 2, we conclude that Condition 3 ⇒ Condition 1.
Revised: 5) By Lemma 2, we conclude that Condition 3 ⇒ Condition 1.

Original: Here we prove that Learnability in D µ,bXY for H ⇒ Condition 3.
Revised: Here we prove that Learnability in D µ,bXY for H ⇒ Condition 3.

Original: Since D µ,bXY is the prior-unknown space, by Theorem 1, there exist an algorithm A : ∪ + ∞ n =1 ( X × Y ) n → H and a monotonically decreasing sequence ϵ cons ( n ) , such that ϵ cons ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D µ,bXY ,
Revised: Since D µ,bXY is the prior-unknown space, by Theorem 1, there exist an algorithm A : ∪ + ∞ n =1 ( X × Y ) n → H and a monotonically decreasing sequence ϵ cons ( n ) , such that ϵ cons ( n ) → 0 , as n → + ∞ , and for any D XY ∈ D µ,bXY ,

Original: [Equation]
Revised: [Equation]

Original: Then, for any ϵ > 0 , we can find n ϵ such that ϵ ≥ ϵ cons ( n ϵ ) , therefore, if n = n ϵ , we have
Revised: Then, for any ϵ > 0 , we can find n ϵ such that ϵ ≥ ϵ cons ( n ϵ ) , therefore, if n = n ϵ , we have

Original: [Equation]
Revised: [Equation]

Original: which implies that there exists S ϵ ∼ D n ϵ X
Revised: which implies that there exists S ϵ ∼ D n ϵ X

Original: I Y I such that
Revised: I Y I such that

Original: [Equation]
Revised: [Equation]

Original: Therefore, for any equivalence class [ D ′ XY ] with respect to D µ,bXY and any ϵ > 0 , there exists a hypothesis function A ( S ϵ ) ∈ H such that for any domain D XY ∈
Revised: Therefore, for any equivalence class [ D ′ XY ] with respect to D µ,bXY and any ϵ > 0 , there exists a hypothesis function A ( S ϵ ) ∈ H such that for any domain D XY ∈

Original: [ D ′ XY ] ,
Revised: [ D ′ XY ] ,

Original: [Equation]
Revised: [Equation]

Original: which implies that Condition 3 holds.
Revised: which implies that Condition 3 holds.

Original: Therefore, Learnability in D µ,bXY for H ⇒ Condition 3.
Revised: Therefore, Learnability in D µ,bXY for H ⇒ Condition 3.

Original: 7) Note that in 4), 5) and 6), we have proven that
Revised: 7) Note that in 4), 5) and 6), we have proven that

Original: Learnability in D µ,bXY for H ⇒ Condition 3 ⇒ Condition 1, and Learnability in D µ,bXY for H ⇔ Condition 1, thus, we conclude that Learnability in D µ,bXY for H ⇔ Condition 3 ⇔ Condition 1.
Revised: Learnability in D µ,bXY for H ⇒ Condition 3 ⇒ Condition 1, and Learnability in D µ,bXY for H ⇔ Condition 1, thus, we conclude that Learnability in D µ,bXY for H ⇔ Condition 3 ⇔ Condition 1.

Original: 8) Combining 4) and 7), we have completed the proof.
Revised: 8) Combining 4) and 7), we have completed the proof.

Original: M.2 Proof of Theorem
Revised: M.2 Proof of Theorem

Original: Theorem 12.
Revised: Theorem 12.

Original: Let K = 1 and the hypothesis space H be score-based ( H = H σ,λ q ,E , where E is in
Revised: Let K = 1 and the hypothesis space H be score-based ( H = H σ,λ q ,E , where E is in

Original: Eqs.
Revised: Eqs.

Original: (5) or (6) ) or FCNN-based ( H = H σ q ) .
Revised: (5) or (6) ) or FCNN-based ( H = H σ q ) .

Original: Given a prior-unknown space D XY , if there exists a domain D XY ∈ D XY , which has an overlap between ID and OOD distributions (see Definition 4), then OOD detection is not learnable in the domain space D XY for H .
Revised: Given a prior-unknown space D XY , if there exists a domain D XY ∈ D XY , which has an overlap between ID and OOD distributions (see Definition 4), then OOD detection is not learnable in the domain space D XY for H .

Original: Proof of Theorem 12.
Revised: Proof of Theorem 12.

Original: Using Proposition 1 and Proposition 2, we obtain that inf h ∈H R in D ( h ) = 0 and inf h ∈H R out D ( h ) = 0 .
Revised: Using Proposition 1 and Proposition 2, we obtain that inf h ∈H R in D ( h ) = 0 and inf h ∈H R out D ( h ) = 0 .

Original: Then, Theorem 3 implies this result.
Revised: Then, Theorem 3 implies this result.

Original: Note that if we replace the activation function σ (ReLU function) in Theorem 12 with any other activation functions, Theorem 12 still hold.
Revised: Note that if we replace the activation function σ (ReLU function) in Theorem 12 with any other activation functions, Theorem 12 still hold.