Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning
Authors: Xiaoqian Wu
Yong-Lu Li
Jianhua Sun
Cewu Lu
Venue: NeurIPS 2023 poster (2023)
Keywords: neuro-symbolic
visual reasoning
human activity understanding

Abstract: Human reasoning can be understood as a cooperation between the intuitive, associative "System-1'' and the deliberative, logical "System-2''. For existing System-1-like methods in visual activity understanding, it is crucial to integrate System-2 processing to improve explainability, generalization, and data efficiency. One possible path of activity reasoning is building a symbolic system composed of symbols and rules, where one rule connects multiple symbols, implying human knowledge and reasoning abilities.
Previous methods have made progress, but are defective with limited symbols from handcraft and limited rules from visual-based annotations, failing to cover the complex patterns of activities and lacking compositional generalization. 
To overcome the defects, we propose a new symbolic system with two ideal important properties: broad-coverage symbols and rational rules. Collecting massive human knowledge via manual annotations is expensive to instantiate this symbolic system. Instead, we leverage the recent advancement of LLMs (Large Language Models) as an approximation of the two ideal properties, i.e., Symbols from Large Language Models (Symbol-LLM). 
Then, given an image, visual contents from the images are extracted and
checked as symbols and activity semantics are reasoned out based on rules via fuzzy logic calculation.
Our method shows superiority in extensive activity understanding tasks. Code and data are available at https://mvig-rhos.com/symbol_llm.

Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning
Xiaoqian Wu
Yong-Lu Li
Jianhua Sun
Cewu Lu"
Shanghai Jiao Tong University fenlighten, yonglu_li, gothic, lucemu)dsjtu.edi edu.
Abstract
Human reasoning can be understood cooperation between the intuitive, asso- ciative "System-1" and the deliberative, logical "System-2".. For existing System- 1-like methods in visual activity understanding. it is crucial integrate System-2 processing to improve explainability, generalization, and data efficiency. One possible path of activity reasoning is building symbolic system composed of symbols and rules, where one rule connects multiple symbols, implying human knowledge and reasoning abilities. Previous methods have made progress, but defective with limited symbols from handcraft and limited rules from visual-based annotations, failing cover the complex patterns of activities and lacking com- positional generalization. To overcome the defects, we propose a new symbolic system with two ideal important properties: broad-coverage symbols and rational rules. Collecting massive human knowledge via manual annotations is expensive instantiate this symbolic system. Instead, we leverage the recent advancement of LLMs (Large Language Models) as approximation of the two ideal properties, i.e., Symbols from Large Language Models (Symbol-LL.M), Then, given an image, visual contents from the images are extracted and checked symbols and activity semantics are reasoned out based on rules via fuzzy logic calculation. Our method shows superiority in extensive activity understanding tasks. Code and data are available lhtps:::/// / //vv//vvv--/gg///ee seriiggrhose rhos, cww/ symbol _11m.
Introduction
Human reasoning can be understood as cooperation between cognitive systems: the intuitive, associative "System-1" and the deliberative, logical "System-2" [25]. An example of System-2 reasoning is illustrated in Fig. Th. feather and an iron ball are dropped the same height in vacuum, which one will land first? Human intelligence derives the answer from the Universal Gravitation Law, i.e., reasoning in physical symbolic system avoid intuitive errors. As a critical component in building Al systems, visual activity understanding urgently needs System-2 reasoning.
Previous works (26 17 30 19 typically establish the mapping between visual inputs and natural language, where knowledge is embedded into some non-symbolic form such the weights of language models [24) Though effective, these System- 1-like methods depend on large-scale visual data, thus suffering from diminishing marginal utility. Also, it fails in explainability and generalization due to black-box computing. Thus, is important to integrate System-2 reasoning.
With one glance at an image, we can effortlessly ground visual inputs to symbols/concepts and apply commonsense reasoning to imagine the world beyond the pixels, process similar to the gravity case. Fig. Tp shows a possible path for System-2 visual reasoming for human activity: a symbolic system composed of symbols and rules. One rule "Hip seated engine A Hands hold onto the side of boat - Human ride a boat" connects symbols
boat Hands control an
"Corresponding authors.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

feather
iron ball
System-2 Reasoning
A Asttract
= Mmm
mo
d1 d2
Answer
a
Which will land first?
b)
Activity Reasoning
bstract
my: Hip seated in a beat. myt Hands control
Human a boat
Activity Semantic
an engine
mat Hands onlo the side of boat
Rule: m, A m2 A mg t=
System-2 reasoning in a) solving physical questions and b) understanding human activities. They both involve a symbolic system implying human commonsense.
Figure 1:
"Hands control an engine" "Human ride a boat") and implies human commonsense. Then, given an image, symbols are first extracted from visual inputs and then activity semantics are deduced based on rules. In this paper, we focus on this line of System-2 reasoning and zoom in from general visual understanding to human activities in various complex contexts, which is a challenging problem in visual understanding.
Current activity reasoning methods are not well equipped with System-2 reasoning. For instance, HAKE (21 20) adopts a similar path as Fig- from the perspective of human body part states and makes progress toward activity reasoning, but is defective with limited symbols from handcraft (93 primitives) and limited rules from visual-based annotations (100 K+ images). For example, Fig. shows several rules related to the activity "Human ride a boat", e.g.,
green) Hip seated in a boat A Hand hold onto the side of the boat A Hand operate the steering wheel } Human ride boat (R2, orange) A rope is attached to a boat A Hand hold the rope A Hand adjust sail Feet hold onto the rope A Leg stretched out -= Human ride boat
However, the rules summarized by (0 are "(R3) Hip sit in A Hand hold |= Human ride boat" "(R4) Hand hold A Feet close with + Human ride a boat", which fail to cover the complex patterns human activities. Thus, it lacks compositional generalization, i.e., a generalized understanding of visual concepts, their relationships. and their novel combinations.
In view of this problem, we propose a novel symbolic system that implies abundant human knowledge and organizes diverse reasonable rules as in Fig. Tb. To achieve this, the symbols should be with broad semantic coverage express various activity contexts. Moreover, the rules should be rational and unambiguous. R4 is a failure case, where 'Hand hold" and "Feet close with" do necessarily lead "Human ride a boat", and "Human tie boat"""uma"" exit boat" are other possible conclusions. Ambiguous semantics should be corrected in rules to avoid confusion. Collecting massive human knowledge via manual annotation is expensive instantiate this symbolic system. Our main insight is Symbols from Large Language Models (Symbol-LLM)), i.e., leveraging the recent advancement of Large Language Models (LLMs) approximate the above two important properties. This is achieved by the symbol-rule loop and entailment check strategy. Then, the symbolic system can be used to reason with visual inputs, where visual contents from the images are extracted and checked as symbols and activity semantics are reasoned out via fuzzy logic calculation.
In conclusion, our main contributions are: 1) We point out that current System-2 reasoning (20) suffers from a defective symbolic system with hand-crafted symbols and limited, ambiguous rules. 2) To overcome the defects, we propose a novel symbolic system with broad-coverage symbols and rational rules. Further, we propose Symbol-LLM to instantiate it and show how it helps visual reasoning. 3) In extensive activity understanding tasks, our method shows superiority in explainability. generalization, and data efficiency.
2 Related Work
22 10 is still a difficult task compared with object detection 28 of pose estimation [9]. This is caused by its complex visual patterns and long- tail data distribution [32
Activity Understanding

---

There are mainly image-based S 151, video-based [4] 4 and skeleton-base B1 7 methods. In this paper, we focus on image-based methods.
Vinuan-L Models (VLMs) have been intensively studied recently. Trained with web-scale image-text pairs, they can make zero-shot predictions various recognition tasks 37 including human activities. Though effective, previous System-l-like methods [26] 17
30 [9] need to be
improved in explainability, generalization, and data efficiency. Thus, integrating System-2 reasoning. which is our main focus, is important.
Probabilistic Programming [27[1] [T] unifies probabilistic modeling and traditional programming. We follow the principles of the standard first-order logic (FoL) in probabilistic programming, e-g., logical connectives A, V. The standard FoL typically solves simplified relational tasks, where symbols are definitely True/False. However, when adapting Fol. in vision reasoning tasks, the symbols are not known to be True/False. Instead, the models required to predict the symbol probabilities (usually via a neural network). Thus, a hybrid "neuro-symbolie" method is adopted.
Neuro-Sy Symbolic Reasoning provides possible way of combining System-1 I and System-2 2 learning, where knowledge is represented in symbolic form and learning and reasoning are computed by neural network.
[13. Recently, researchers utilize neural-symbolic reasoning in tasks including visual question answering 22 16 34 24 29], decision-man making 6 motion programming [18], scene understanding [23). Focused activity reasoning. HAKE [21]200 propose intermediate space spanned by human body part states as primitives. To overcome its inherent defect in compositional generalization, we propose novel symbolic system with broad semanties and rational rules.
3 Method
3.1 Formulating the Symbolic System
3.1.1
Structure
We first analyze the structure of the symbolic system. As aforementioned, the activity symbolic system is composed of rules that connect symbols, which implies human knowledge and reasoning abilities. Formally, one rule involves several symbols, where one of them is the conclusion and the others are premises. Each rule takes the form my A m2 A... A M., F where M {m, l==1 the premise set and c is the conclusion. Each conclusion corresponds to more than one rule with varied combinations of premise symbols because one activity typically has different visual patterns. Multiple rules for one conclusion are logically connected with V, i.e., one conclusion holds if at least rule is satisfied. hyper-graph is a proper data structure to handle the complexity of the connection. Therefore, we define the structure of the activity symbolic system as a B-Graph [12].
Definition 1. (Directed Hypergraph (3) A hypergraph is generalization of a graph in which an edge can join any number of vertices. A directed hypergraph a pair (X, E), where X is set of vertices, and & is a set of pairs of subsets of X. Each of these pairs (D, C) E E is hyperedge; the vertex subset
its domain, and its codomain.
Definition 2. (B-Graph [12) B-graph a type of directed hypergraph with only B-ares. B-arch is a rype of a hyperedge that is directed to a single head vertex, and away from all its other vertices.
Definition 3. (Activity Symbolic System) The activity symbolic system (S, R) is B- B-Graph, where S is a set of vertices/symbols, and R is set of pairs of subsets of S. Each of these pairs r= r-MM, C*) E RR is hyperedge/rull; the vertex subset M { is its domain/premises, and C = {c} is its codomain/comclusson Equivalently, rule takes the form r N m F
Since [C1 = 1, r is a B-arch.
Definition 4. (Decomposition of Activity Symbolic System) One conclusion e² corresponds to one symbolic sub-system (S.-. RR.) where S.- C S, Re C R. Vr (M,C) E Re=, C {c'). vs S.- (cc), 3r (M,C) E Ro,, S E M.
Def. 3 is depicted in Fig. Def.2 is based on Def. 22 and analysis above. In applications, we typically judge specific conclusion with other symbols/rules removed. is achieved by decomposing the activity symbolic system (graph) into sub-systems (sub-graphs) in Def. 4
3

---

b) Semantic Coverage
Approximation
Symbol-Rule Loop
Rules:
Image
Language
O OF
Semantics
Sementics
hoad
Queue
O
OOOO
Symbol Samantics
Symbol
pop
push
Sementics
Rule Extension
Decomposition
Logical Entailment
A Approximation
Entailment Check
Entailment
A nierr
X
((r)
ch
LLM
picuuee
Estimnate
Entailment ScOrO
Figure 2: Our activity symbolic system. a) Structure and decomposition of the symbolic system (Sec. (.I.I. b) Semantic coverage (Sec. 3.1.2p. It can be approximated based on LLMs knowledge and achieved via the symbol-rule loop (Sec. 3.2.1) c) Logical entailment (Sec. 3.1.2 It can be approximated based on an entailment scoring function and achieved by entailment check (Sec. 3.2.2)
3.1.2 Two Ideal Properties
Existing methods [21 20 can also be formulated under this structure. However, they f'ail cover the complex patterns of activities and lack compositional generalization. To overcome the defects, we propose two important ideal properties: 1) symbols should be with broad semantic coverage to express different conditions in the activity image database; 2) rules should satisfy logical entailment [8] add rationality and avoid ambiguity. i.e., the premises set should lead the conclusion without exception. They are formulated in Def. 5 and illustrated in Fig. D.c.
Definition 5. (Semantic Coverage of Activity Symbolic System) Given a very large-scale activity images database DD ((1, Ar, St)} (I: image, Ar: ground-truth activities happening in I, Sf ground-truth symbols happening in T), then V(I, Ar,St) E D, Vs E S₁, 8 E S.
Definition (Logical Entailment of Activity Symbolic System) Vr = (M, C) E R, M
{m; 1a have:
1. V(I, Ar, D, if C S,, then e Ar;
2. v1 <i < n.. n,, Ar, E D, M, S₁, but $ Ar.
3.2 Instantiating the Symbolic System
To instantiate the proposed symbolic system, it is expensive to collect massive human knowledge via manual annotation as in HAKE [0]. Instead, use LLMs which have profoundly reshaped the acquisition of human knowledge and shown impressive language reasoning capabilities, i.e., Symbol-LLM framework.
Sec. 3.2.1 and Sec. 3.2.2] discuss the approximation of the two ideal properties with the help of LLMs and provide practical solutions, then summarized pipeline is presented in Sec. 3.2.3
3.2.1 Solving Semantic Coverage
Approximation. In Def. 5 the target domain of semantic coverage is very large-scale activity image database, which is too costly to implement. For instance, HAKE (1] collect symbols from 100 K+ image via manual annotations, which is heavy workload but still very limited. Here, the target domain can be replaced with semantic coverage of language models since natural language is another carrier of human commonsense knowledge. LLMs profit from large-scale text pre-training and imply abundant text-based human knowledge, thus can handle complex visual activity patterns without extra visual data. The approximation of Def. is:
Definition (Approximation of Semantic Coverage of Activity Symbolic System) Given LLM C and activity set A. v A E A, & implies symbol set SA premises of A, then Vs E SA.S E S.

---

Symbol-Ruke Loop. In the implementation, generating rules connecting the symbols is non-trivial and challenging. Intuitively, we can ask LLMs "What is the rule of the activity". However, the instruction is ambiguous for LLMs to generate satisfying answers. More information should be provided to generate rules. Also, it is costly to generate all symbols and exhaustively query their connections. Thus, we propose to adopt symbol-rule loops to generate them alternately. Given known symbol, we can add another symbol and extend it into a rule via prompting:
(Rule Extension)
In picture, IF [<known symbols>] AND [condition] THEN [<activity>]. [condition] one concise phrase. The format is "<The person's hands/ arms httmm/////lll///ettttt <verb> <object>". What is [condition]?
Then, the answer for "[condition]" is the extended symbol. Also, a candidate rule "<known symbols> A <extended symbol> - <activity>" is generated. Thus, we get new extended symbol from this known symbol, and they are connected with rules. The new symbol can be repeatedly used as a known symbol to generate new rules. As is shown in Fig- [2.. a known symbol (the brown circle) is first "popped out" from queuc, and new symbols (the green circle) and rules (the pink edge) are generated by extending the known symbol. Then, the new symbol is "pushed in" to start the next loop. Thus, LLMs' implicit knowledge is fully exploited to satisfy Def. 7 To get the initial symbols, we ask an LLM to describe hand-related states related to the activity. The prompts mentioned above determined via trial-and-erroo process by human experts randomly sampled activities (detailed in supplementary).
3.2.2 Solving Logical Entailment
Approximation. In Def. 6 it is costly to verify the logical entailment upon a very large-scale activity image database. Instead, we develop an entailment scoring function from an LLM based on its knowledge and language reasoning capability. As illustrated in Fig. 2 as the rule extends from mi } c, my A Amg F e my Am2 A my } c, new symbols are successively added, and the measured entailment score increases. With the scoring function, the approximation of Def. 6
Definition 8. (Approximation of Logical Entailment of Activity Symbolic System) Given
function T-) to measure the entailment score of a rule, and an entailment threshold ej, then Yr (M,C)= E R, M = {mm ii we have:
1. T( mEM = c) 2 Chi
Si nr. T T(AmEMMmL m. c) < Ch.
Entailment Check. As is shown in Fig. 2c, to implement the scoring function based on an LLM, we rewrite the rule sentence and design the prompt as:
(Entailment Check)
picture, <symbol 1>, <symbol probability that he is <activity> at the same time. Choose from: (a)
<symbol n,, Estimate the
0.1, (b) 0.5, (c) 0.7, (d) 0.9, (e) 0.95, (f) unknown.
Then, the output answers can be used the entailment score T(r). whose credibility depends on the knowledge from the LLM. For one rule, the answering text is sampled five times and their average scores are taken as the final result add stability. In practice, we set e₁ 0.9.
3.2.3 Summarized Pipeline
The pipeline of the symbolic system can be summarized as: given a target activity set A, or equiva- lently, a conclusion set C, we generate a sub-system (S., R.) for each C (Def. 4.. The symbolic system is merged as (S, R) = (Urec S., Usec R.). Alg. shows how to get the sub-system (S,, R.) for given conclusion c. Details are explained in Alg. comment and the supplementary.
3.3 Reasoning with Visual Inputs
With the activity symbol system constructed, we can utilize it to reason with visual inputs. is shown in Fig. visual information is extracted and checked based on the defined symbols and
5

---

Algorithm Instantiating the Symbolic System
Input: conclusion entailment threshold En.
Output: Symbolic Sub-System (Se; Re)
1: Sh t- -Symool Initialization (c)
2: Scand ppssh(.))
b. A queue Sound stores symbols
3: Se - (c), R. 0
4: while not Scand, is ( do
5: Mhko Scand pop()
A known symbol Mino is taken for rule extension The premises set M is set for the current rule
6: M = {mEno)
7:
CacialmmentChhck(AMMM while Entailment Check (M) <
do Mnew <- Rule Aattenioo(MM,) M = M U {mnow}
8:
9:
> A new symbol Mnew is added
10:
end while
11:
Se 4 S.UM
D The symbol set S. is updated D The rule set Re is updated > M is added to Sound with redundant symbols removed
12:
& fr: :MEM m c}
13:
Soand pppppphhMMMM (M)))))
end while
15:
return (Se, Ro)
m
(2)
Symbol Extractor
Symbol Checker
Fuzzy-Logic Calculation
ml
0.9
0.5
0,9
0.7
(+:) min(?:)
0.8
0.8
(v) max
Human ride a boat?
m
O e
m (a) Hands operale the steering wheel
0.7
0.3
0.5
V
mz ( Hip seated in boat
O.1
0.1
Activity Sementic
Figure 3: Visual reasoning with the proposed activity symbol system.
organized as probability distribution on the hyper-graph. Then, the activity semantics can be reasoned out based on the rules and the probability of the symbols. We detail the key steps as follows.
Decomposition. Given the input image and the target activity/concllsion c, we first decompose the symbol system to exclude unrelated symbols and rules based on Def. [ Thus, we get the rule set R. = (r-5) } where rule takes the form ,() m A Am2
(5) A... Mmn.)
(i) Fcc
Extracting Visual Symbols. Given an image, to extract visual content and express in a symbolic form, we measure the probability of each defined symbol (Fig. 3) In (20), the visual extractor is customized model trained with annotations, which costly, especially for our symbolic system with broad-range symbols. Instead, we utilize simplified visual extractor based on existing System- I-like VLMs. To determine the probability of symbol, we convert into a text question and query the answer from VLM. For example, given a premise symbol m (i) "Hip seated in a boat", it converted into a sentence "The person's 's hip is seated in a boat. Yes/No?". Used as a scoring
(f) (1) Then function [ the language model outputs the probability of answering "Yes", "No" as Pn.i:
the probability p!") of the symbol can be obtained via normalizing the Yes/No answer as
(I)
P
P
Checking Visual Symbols. To ensure the reliability of the symbols' probabilities, we developed a checker module to filter out visual symbols with uncertain predictions. Each symbol is paraphrased into several variants, and the predictions of variants should be similar due to semantic consistency. Thus, the standard deviation of the predictions can be used to filter symbols with uncertainty.
Reasoning. With the symbol extractor and checker, the probability p; for each premise symbol mi in the rule set R, is measured. Then, the probability Pr. of the conclusion is reasoned based on premise symbols' probabilities and the rules. Premise symbols within a rule are connected with A, while rules for conclusion connected with V. We utilize the fuzzy-logic calculation [35), where

---

Round1
Round2
Roundi
Rounds 5
RounD Rules 1-6-77
Round3 Rules
Round4 Rules
Round5 Rules: eady stop
6 + omitted
11 14;
2-+ 2-1--1-9
7
12-1
4-+10-6 5-11
8 12; 10: early stop
16;
Figure 4: Detailed process for instantiating the symbolic sub-system of the activity "Human board an airplane". As is shown the left bottom box, there is some difference in graph presentation between this figure and Fig. T3 here the edge connects premises instead of connecting premise and conclusion. The numbers 1-17 correspond to the 17 symbols listed in Tab. [ The in
"1-+6 +1-+6-77-2 7 means that 6, 7, 2 is obtained via I's rule extension after another.
Symbols
(I) hold boarding pass
(2) place luggage in overhead compartment
Round1
(3) adjust seathelt
(4) wave goodbye to loved ones
(5) grip a luggage handle
(6) walk towards the boarding gate
(7) luggage visible beside him
Round2
(8) boarding pass is scanned by airport staff
(9) stand on the jet bridge
(T0) luggage is loaded onto plane
(11) reach for the airplane door handle
(12) stand in line with carry-on luggage
Round3
(13) hold the carry-on luggage
(14) open the airplane door
(15) move forward in the line
Round4
(16) move towards the airplane door
Round5
(17) airline staff checking the boarding pass
Table
Generated symbols & rules for "human board an airplane". Round i: i-th symbol-rule loop.
the operator A y, V are replaced with mán(x,,), max(x, v) respectively. Thus, we have
P. = maz (minssssn ()))
The fuzzy-logic calculation is more explainable and lightweight than the logical modules in 20 whose parameters need extra training samples. Benefiting from the sound definition our symbolic system, it has the potential to achieve better performance even with simplified reasoning calculations.
4
Experiment
4.1 Dataset and Metric
We conduct experiments on image-level activity understanding benchmarks with diverse tasks:
HICO (5) (Human-Object Interaction (HOI) recognition), Stanford40 (33] (action recognition),
HAKE 21 Verb (verb recognition), HAKE (21 -PSSaa (conditional PaSta Q-A). HAKE [21]-Vrbb
and HAKE PaSta is the newly constructed benchmark based on HAKE 21 data. We adopt mAP metric for HICO (5), HAKE [1 -erb and Stanford40 [33), and top-1 accuracy metric for HAKE [1] For more details, please refer to the supplcmentary.
Symbolic System Experiment
An example. The LLM used is OpenAl GPT3.5. To demonstrate the instantiation of our symbolic system, we take the activity "human board an airplane" as an example. Tab. shows the symbols and rules generated, and Fig. Tig.-illi illlustrates the detailed process. In the 1st round, 5 initial symbols are generated. Then in the 2nd round, they are extended into rules satisfying entailment. Later, the new symbols used in the next round. Finally, 17 symbols and 12 rules are generated within
7

---

a)
b)
20
symbals count
rules count
Entailment Score
0.8
15
Count
0.6
Rule 0
Rule
10-
0.4
Rule 1
Rule
0.2
0.0
1
2
5
Round
#Premise Symbols in the Rule
Figure 5: Statistics of the symbolic system. a) Accumulated symbols rules count in each round. The data is from 50 randomly sampled activities. The average value and fluctuation range are shown. b) Increased entailment scores as the 10 randomly sampled rules extended with more premise symbols.
rounds. In practice, at most 15 symbols are used for rule extension avoid semantic redundancy. For example, in Fig. [ symbols 16 and 17 are not used for rule extension, leading to a stop in the 5th round. From Fig. 4 and Tab. [I, we can find that diverse symbols are mined, including corner cases such as "adjust seatbelt" "reach for the airplane door handle", verifying the effectiveness of the proposed symbolic system. Some special cases are shown as gray text. The rule "6-+1-77-8" is equivalent to the existing rule "I 1--6-77-8" thus omitted. Extension from the symbols 10 and 14 stops early because adding premises nevertheless leads to drop in entailment scores. In practice, multiple rules are extended from a known symbol to increase diversity.
Entailment. Fig. shows statisties of the symbolic system. In Fig. 5 randomly sample 50 activities and show the accumulated symbols and rules count in each round, verifying the diversity of the generated symbols and rules. In Fig. 5p. we randomly sample 10 rules and show the entailment scores change as the rule extends with more premise symbols. For example, for rule O: mı AmyAmaA ma }= c, the entailment score for mi - c, m/ Am2 } C, mi Amgg }= C, 18 Amig Aarii//////ii Am e are 0.28, 0.86, 0.87, and 0.93, respectively. The score is calculated by querying 5 times and averaged. We find a climb up the entailment score, implying the effectiveness of the entailment check. Once the entailment score surpasses the threshold ед 0.9, it is regarded as rule equipped with logical entailment and updated into the symbolic system.
Performance Evaluation. To evaluate the symbolic system, we construct a SymAct (Symbol Activity) test set. It small subset of the HICO test set (120 images for activity classes). For each image and each of its ground-truth activities, we list all of the symbols related to the activity in the symbolic system and annotate whether a symbol happens (0/1) in the image via human judgment. We find the proposed symbolic system has: 1) broader semantic coverage than HAKE. For quantitative measurement, we count the happening symbols for each image-activity pair under different symbolic systems. The average number is 10.8 for ours and 1.8 for HAKE. 2) more rational rules than HAKE. We evaluate confusion by counting different image-activity pairs which share the same symbols. For the HAKE symbolic system, the confusion problem severe, with 261 confusion pairs over Ci = 7140 pairs (accounting for 3.7%). For our symbolic system, there are no confusion pairs on the test set because of the presentation ability of symbols and entailment check.
Robustness. We analyze the robustness of the generated symbolic system. We find satisfies: 1) Convergence. After certain stopping criterion, a new round will bring few or even no new symbols the sub-system. practice, the stopping criterion is: At most 15 symbols are used for rule extension to avoid semantic redundancy. After that, new symbols are not used for rule extension, leading to stop the loop. 2) Low sensitivity to initial conditions. The symbol-rule loop provides reliable and stable symbolic sub-system over range of initial conditions. 3) Low sensitivity to different prompts. Experiment results are less sensitive to prompts with minor differences, i.e., slightly paraphrasing the prompts. For more details, please refer to the supplementary.
4.3 Visual Reasoning Experiment
Implementation and Settings. For visual reasoning, we use BLIP2 19 ViT-g FlanT5-XI. model to extract visual symbols. BL.IP2 [19] is visual-language pre-trained model with a frozen large language model, thus well equipped with question-answering abilities to effectively extract symbols.

---

Symbol probability
11
III
IV
my talk with seller
0.87
0.07
0.16
mg: reach for an orange
0.14
0.68
0.95
0.12
mg: seller hand over orange
0.94
0.82
0.09
0.13
Rules for buy orange
max(min(0.92,0014
my: stand in front of fruit stand
0.57
0.85
0.04
0.77
11: my A m2 I= c,
(0.900,,,)
mg: place orange bag
0.67
0.90
0.06
0.45
min (0.57,0.67,00668)
12: my My
mg: pick orange from a basket
0.68
0.83
0.03
0.74
mg: A My A M2 F C,
min((0.57.0047.0..6))
T4: m4 my A mg F
my: hold bag of oranges
0.47
0.88
0.05
0.75
min (1..(0.57.0.3,00.82))
Tis: m4 My A M20 F c
ma: reach Ior wallet
0.86
0.83
0.04
0.43
my: seller put the orange in bag
0.86
0.04
0.45
0.92
0,85
0.07
0.45
mao: give money to seller
0.82
0.87
0.04
0.41
Figure 6: Visualization results. Symbols & activity predictions for "human buy orange" are shown.
HICO mAP fine-tuned
HAKE-verb mAP zero-shot
Stanford-40 zero-shot mAP
HAKE-PaSta zero-shot Acc(%) (%)
Method
zero-shot
fine-tuned
CLIP (26
67.12
37.08
73.82
43.92
75.68
39.36
CLIP [26 Reasoninn
69.73
43.21
75.27
48.95
82.22
40.47
BI.IP 19
50.61
49.47
91.85
43.81
BLIP2 19
+Rcasoning
53.15
51.37
92.59
44.65
Table 2: Results on activity benchmarks. More HICO [ baselines are listed in supplementary.
As a plug-and-pley the reasoning is compatible with existing System- I-like methods. We follow the setting of HAKE [20] to integrate the reasoning result with the prediction of System- --liee methods. We mainly choose 2 typical gystem-l-like methods: 1) CLIP [26]: a visual-language model pre-trained with lcrge-scale image-text pairs and contrastive objective, where use its ViT-L/14 model with a resolution of 336 pixels; 2) BLIP2 [9]: with its image-to-text generation ability, functions either as a System-1-lite ability of the models, we adopt both "fine-tuned" and "zero-shot" settings, where the former model is fine-tuned on the training set of the dataset.
baseline, or the symbol extractor for reasoning. To fully test the
The results of visual reasoning are shown in Tab. 2 Comparing the two baselines, we find that with frozen language model, BLIP2 [19] is more capable of understanding activity semantics and outperforms CLIP [26] in various zero-shot benchmarks. Under the ine-tuned setting, our method achieves 69.73 and 75.27 mAP respectively
Results.
HICO (5) and HAKE [1 -erb, which outperforms the baseline CLIP [26] with 2.61 and 1.45 mAP respectively, verifying its effectiveness. Under the zero-shot setting, our method also outperforms existing System- -liike methods. The improvement compared with CLIP 266 is: 6.13 mAP (HICO (5). 5.03 mAP (HAKE (err-Verb), 6.54 mAP (Stanford40 (33)), 1.11% % acc (HAKE (0 PaStaa) The improvement compared with BLIP2 19 is: 2.54 mAP (HICO (5). 1.90 mAP (HAKE 20)-Ver),, 0.74 mAP (Stanford40 (33)) 0.84% ace (HAKE -aStaa... In conclusion, our method brings performance improvement and shows significant zero-shot generalization abilities in various activity understanding tasks. Notably, the improvement is achieved by exploiting knowledge instead of adding visual training samples. The exploited knowledge can be reused for future study. Thus, our method is superior in data efficiency. Further, we visualize the results Fig. 6 The rules and the logic calculation add explainability the inference process. Also, they help to correct false negative (I, II in Fig, 6) and false positive (III, IV) cases, thus improving compositional generalization.
4.4 Ablation Study
We conduct ablation studies on HAKE 20 Verb zero-shot setting and list the results in Tab. 3
Reasoning, We first verify the effectiveness of integrating System reasoning. Results of CLIP 26 baseline can be either combined with reasoning results or trivially combined with other baseline results (CLIP (26 +BLIP2 [9). We find that the former is superior (48.95 mAP) to the latter (44.76
mAP), though both outperform the baseline CLIP 26 (43.92 mAP). It verifies the necessity of System-2 2 reasoning other than trivially combining predictions from two models.

---

Symbolic System. We then verify the effects of the symbolic system on visual reasoning. We
adopt the symbolic system in HAKE [20] and find degradation from 48.95 to 44.98 mAP,
as its defects in semantic coverage and rule rationality, and the short-
age is amplified the fuzzy logic calculation without learnable pa- rameters. Next, we construct the symbolic system without logical entailment, where each rule has premise symbols without checking its entailment score. It leads to dropped performance because of rules with errors or incomplete semantics. Then, the symbol -rule loop is removed, with only rules in the 1st and 2nd rounds updated to the symbolic system. As the diversity of rules decreases, the model suffers from a decreased performance. The importance of diversity further verified by the performance drop when only 80%, 50%, and 20% randomly sampled rules are used.
Method
mAP
CLIP
26
Reasoning
48.95
CLIP
20 BLIP2
19
44.76
CLIP
43.92
rules from HAKE 20
44.98
w/o cutailment check
46.51
wo symbol-rule loop
47.53
80% rules
48.57
50% rales
47.61
20% rules
46.22
CLIP (6) extractor
w/o checking symbols
48.17
Table 3: Ablation studies on zero-shot HAKE [20]- Verb.
Symbols Extractor/Checker Next, we replace the symbol extractor [9] with CLIP (26] and find performance fall with weaker ability to extract visual information. Also, reasoning without a symbol checker suffers from degradation due to the negative effect of inaccurate symbol predictions.
4.5 Bottleneck Analysis
Based on the SymAct test set, we make further analysis of the bottleneck of System-2 reason- ing. We investigate the impact of symbol prediction and symbolic system separately. With perfect symbol predictions, we assume the probabilities of symbols are definitely known as 0/1 instead of p [0, 1]. The ground-truth symbols are annotated as explained in Sec. 4.2 With perfect symbolic system, we assume that the generated rules cover samples (image-activity pairs). The performance drop (100.00-60.79 mAP, 83.11-44.52 mAP) indicates the symbolic system errors, i.e., the generated rules have not covered all samples (image-activity pairs). The drop (100.00 + 83.11 mAP, 60.79 +41.52 mAP) indicates the
Symhol Prediction
Symbolic System
mAP
perfect
perfect
100,00
imperfect perfect imperfect
perfoct
83.11
imperfect
60.79
imperfoct 41.52
errors of symbol predictions, i.e., symbol probabilities cannot be predicted accurately. In the supplementary, we also provide two failure cases that emerge from symbol prediction errors and symbolic system errors respectively.
Table 4: System- 1/2 analysis on SymAct
test set.
5 Conclusion and Discussion
In this work, we rethink the symbolic system in activity reasoning and propose new one with broad-coverage symbols and rational rules. Thus, enhanced System-2 reasoning is integrated into System-1. We demonstrate how to instantiate and how it helps to reason with visual inputs. Our method shows superiority in explainability, generalization, and efficiency in extensive experiments.
Computation Cost. Symbol predictions will increase the computational cost as trade-off for explainability and generalization. It can be cased by discovering the hierarchical and reusable nature of the symbols, which is detailed in the supplementary.
Broader Application. The method could be more general and facilitate real-world applications. We choose activity understanding as good and important initial test bed because it is difficult task with complex visual patterns and a compositional nature. To verify the broader application, we provide some initial results on object recognition and VCR (36] tasks in the supplementary.
Limitation. Rules generated from language models mostly boost System- 1, but sometimes bring language bias, e.g., the person only decides to buy an orange despite apples in the background. Further, the approximation is mainly based on pre-trained LLMs and can be improved with more elaborate designs, e.g., numan-in-the-loop. the-loop, customized LLMs with higher-quality knowledge. Besides, System- 2 reasoning may be boosted if integrated into the System-1 training process. Also, instance-level activities and temporal encoding are beyond our scope since our main focus is an overall framework to formulate and construct a novel symbolic system. We leave them to future work.
10

---

Acknowledgments and Disclosure of Funding
Some of the drawing materials in Fig. 12 are designed by Layerace / Freepik.
This work is supported in part by the National Natural Science Foundation China under Grants 62306175, National Key R&D Program of China (No.2021ZD0) 10704), Shanghai Municipal Science and Technology Major Project (2021SH2DZZ00102), Shanghai Qi Zhi Institute, Shanghai Science and Technology Commission (21511101200)
References
[1] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as i can, not as say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022.
[2] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In CVPR. 2016.
[3] Claude Berge. Hypergraphs: conbinatorics finite sets, volume 45, Elsevier, 1984.
[4] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kinetics dataset. In CVPR, 2017.
[5] Yu Wei Chao, Zhan Wang, Yugeng He, Jiaxuan Wang, and Jia Deng. Hico: benchmark for recognizing human-object interactions in images. In ICCV, 2015.
Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. Neural logic machines. arXiv preprint arXiv:/904. 1694, 2019.
[7] Yong Du, Wei Wang, and Liang Wang. Hierarchical recurrent neural network for skeleton based action recognition. In CVPR, 2015.
[8] Richard Evans, David Saxton, David Amos, Pushmect Kohli, and Edward Grefenstette, Can neural networks understand logical entailment? arXiv preprint (00.:08002208333 2018.
[9] Hao-Shu Fang, Jiefeng Li, Hongyang Tang, Chao Xu, Haoyi Zhu, Yuliang Xiu, Yong-Lu Li, and Cewu Lu. Alphapose: Whole-hody regional multi-person pose estimation and tracking in real-time. TPAMI, 2023.
[10] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In ICCV, 2019.
[11] Daan Fierens, Guy Van den Broeck, Joris Renkens, Dimitar Shterionov, Bernd Gutmann, Ingo Thon, Gerda Janssens, and Lue De Raedt. Inference and leamiing in probabilistic logic programs using weighted boolcan formulas. Theory and Practice of Logic Programming, 15(3):358-40)) 2015.
[12] Giorgio Gallo, Giustino Longo, Stefano Pallottino, Sang Nguyen. Directed hypergraphs and applica- tions. Discrete applied mathematics, 42(2-3): 1993.
[13] Artur d' Avila Garcez, Marco Gori, Luis C Lamb, Luciano Serafini, Michael Spranger, and Son N Tran. Ncural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv preprint Xrviv: 06088, 2019.
[14] Chunhui Gu, Chen Sun, David A Ross, Carl Vondrick, Caroline Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, al. Ava: video dataset of spatio-temporally localized atomic visual actions. CVPR, 2018.
[15] Saurabh Gupta and Jitendra Malik. Visual semantic role labeling. arXiv preprint 10X.v:1505.04444. 2015.
[16] Drew Hudson and Christopher D Manning. Learning by abstraction: The neural state machine. NeurIPS, 2019.
[17] Chao Jia, Yinfei Yang. Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le. Yun-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling visual and vision-language representation learning with noisy text supervision. In /CML, 2021.
[18] Sumith Kulal, Jiayuan Mao, Alex Aiken, and Jiajun Wu. Hierarchical motion understanding via motion programs, In CVPR, 2021.

---

[19] Junnan Li. Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping waangugge-imagee pre-training with frozen image encoders and large language models. arXiv preprini urXiv:2301. 125299 2023.
[20] Yong-Lu Li, Xinpeng Liu, Xiaoqian Wu, Yizhuo Li, Zuoyu Qiu, Liang Xu, Yue Xu, Hao-Shu Fang, and Cewu Lu. Hake: knowledge engine foundation for human activity understanding. TPAMI, 2022.
[21] Yong-Lu Li, Liang Xu, Xinpeng Liu, Xijie Huang, Yue Xu, Shiyi Wang, Hao-Shu Fang. Ze Ma, Mingyang Chen, and Cewu Lu. Pastanet: Toward human activity knowledge engine. In CVPR, 2020.
[22] Ji I Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In ICCV, 2019.
[23] Yunchao Liu and Zheng Wu. Learning to describe scenes with programs. In JCLR, 2019.
[24] Kenneth Marino, Xinlei Chen, Devi Parikh, Abhinav Gupta, and Marcus Rohrbach. Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-basee vqa. In CVPR, 2021.
[25] Maxwell Nye, Michael Tessler, Josh Tenenbaum, and Brenden M Lake. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. NeurIPS, 2021.
[26] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021.
[27] Luc De Raedt, Kristian Kersting, Sriraam Natarajan, and David Poole. Statistical relational artificial intelligence: Logie, probahility, and computation. Synthesis lectures on artificial intelligence and machine learning, 10(2):1-199, 2016.
[28] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurlPS, 2015.
[29] Botian Shi, Lei Pan Lu, Zhendong Niu, and Nan Duan. Knowledge aware semantic concept expansion for image-text matching. IJCAI, volume 1, page 2019.
[30] Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojeiech Galuba, Marcus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. In CVPR, 2022.
[31] Raviteja Vemulapalli, Felipe Arrate, and Rama Chellappa. Human action recognition by representing 3d skeletons as points a lie group. CVPR, 2014.
[32] Yue Xu, Yong-Lu Li, Jiefeng Li, and Cewu Lu. Constructing balance from imbalance for long-tailed image recognition. In ECCV, 2022.
[33] Bangpeng Yao, Xiaoye Jiang, Aditya Khosla, Andy Lai Lin, Leonidas Guibas, and Li Fei-Fei. Human action recognition by learning bases action attributes and parts. ICCV, 2011.
[34] Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Josh Tenenbaum. Neural- symbolic vqa: Disentangling reasoning from vision and language understanding. NeurlPS, 2018.
[35] Zadeh. Fuzzy sets. Inform Control, 8:338-353, 1965.
[36] Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. From recognition to cognition: Visual commonsense reasoning. CVPR, 2019.
[37] Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: survey. arXiv preprint urXiv:2304.000 00685, 2023.
12


Reviews:
[{"review_id": "7YoUmmwQCO", "reviewer": "Reviewer_sNaH", "summary": "The paper proposes a novel method for visual activity understanding using an enriched symbolic system. The authors focus on enhancing the integration of intuitive (System-1) and logical (System-2) reasoning processes in AI systems. They argue that the current System-2 reasoning methods suffer from limitations such as a dearth of symbols and limited, ambiguous rules, inhibiting the system's ability to capture the complex patterns of activities and limiting its explainability, generalization, and data efficiency.\n\nIn response to these shortcomings, the authors introduce a new symbolic system enriched with a broader set of symbols and more rational rules. This system, underpinned by commonsense knowledge, aims to foster a more comprehensive understanding of human activities visually. To support their work, they leverage the advancements in LLMs and techniques like the symbol-rule loop and entailment check. \n", "questions": "Missing related references:\n- Knowledge Aware Semantic Concept Expansion for Image-Text Matching, IJCAI 2019.\n\nIt would be beneficial to include more latest baselines for comparison.\n", "limitations": "The authors adequately addressed the limitations.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The concept of developing a new symbolic system that combines aspects of intuitive (System-1) and logical (System-2) reasoning is indeed a creative approach. However, the idea of incorporating commonsense knowledge into AI reasoning is not entirely novel, as there have been previous works exploring similar notions.\n\nThe authors put forward a detailed methodology and apply recent advancements such as Large Language Models (LLMs) in their approach. \n\nWhile the paper is mostly well-written, certain parts, especially the description of the symbolic system and its instantiation, could benefit from more elaboration or clearer explanations.\n\nAlthough the paper targets a significant area in AI - the integration of System-2 reasoning into visual activity understanding - it's unclear how much practical impact this specific method would have on the broader field. The lack of comparison with other methods, especially in various real-world applications, makes it difficult to evaluate the actual significance of their work.\n", "weaknesses": "The paper's evaluation approach, while it encompasses multiple datasets, does fall short in providing a comprehensive set of comparisons with current state-of-the-art methods. The selected baselines do not reflect the full breadth of existing approaches, which undermines the claim of superiority of the proposed method. It would be valuable to include comparisons with more recent vision-language models, such as LLaVA, MiniGPT-4, and LLaMA Adapter, to provide a more meaningful assessment of the proposed system's performance.\n\nThe paper does spend a substantial portion of its content introducing the method background. Shifting sections 3.1-3.2 to the appendix could help maintain focus on the main contribution of the paper, while still providing the necessary context for interested readers.\n\nThe explanation of the core methodology is another area that requires improvement. It is not adequately detailed in the introduction, with key aspects of the proposed method encapsulated in a single sentence (lines 57-58). The paper would benefit from a more thorough and step-by-step explanation of the modeling method in the early part of the paper, such as the introduction, which would help readers better understand the unique features and operations of the proposed system.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n## Contribution\n\nThough it is an active research area to incorporate commonsense knowledge into AI reasoning, this paper has its unique contribution toward integrating a rule-based symbolic logic reasoner into visual activity understanding, where explicit and rationale rules are encoded to solve real-world complex visual patterns and improve generalization composition.\n\n## Generality\n\nThe method could be more general and facilitate real-world applications. We choose activity understanding as a good and important initial test bed because it is a difficult task with complex visual patterns and compositional nature.\nThe proposed symbolic system can be generalized to various tasks (e.g., classification, visual commonsense) with a similar formulation. Some examples are shown below:\n\n1. Object classification\n    - For the object \"crocodile\" in CIFAR-100 [1] dataset, one example rule is: long, slightly curved body $\\wedge$ four short legs $\\wedge$ ... $\\wedge$ a long, muscular tail $\\rightarrow$ crocodile\n\n2. Visual commonsense\n    - For one example QA in VCR [2] dataset (shown in attached pdf Fig.1), one example rule is: person on the right shows affectionate gestures $\\wedge$ person on the left engages in conversation $\\wedge$ ... $\\wedge$ The environment is a restaurant $\\rightarrow$ b\n\nUnder this formulation, semantic coverage and logical entailment remain two important properties under a more general setting, and we can reason with visual inputs via a similar pipeline. \nWe conduct experiments with two zero-shot benchmarks: \n\n1. CIFAR-100 [1] test set. \n\n| Method           | CIFAR-100 Acc (\\%) |\n| -----------      |  -----------        |\n| CLIP             | 77.50               |\n| CLIP+Reasoning   | 78.31               |\n\n\n2. VCR [2] val set. BLIP2 is adopted as a baseline as it is a VQA-style task. VCR val set has 26534 image-question pairs, where we select a subset of 557 pairs for the test. The selected pairs are highly related to human activities (instead of role, mental, .etc) and only involve *one* person's activity. \n\n| Method            | VCR Acc (\\%)|\n| -----------       | ----------- |\n| BLIP2             | 46.32        | \n| BLIP2+Reasoning   | 47.08        | \n\nWe can see that the proposed pipeline can be applied to general visual tasks and facilitate reasoning. In this paper, we focus on method design instead of task generalization.\nWe will add these explanations in the final version and analyze more general settings in future work.\n\n[1] Krizhevsky, Alex, and Geoffrey Hinton. \"Learning multiple layers of features from tiny images.\" (2009): 7.\n\n[2] Zellers, Rowan, et al. \"From recognition to cognition: Visual commonsense reasoning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n\n## Evaluation Baselines\nWe respectfully explain that these more recent vision-language models (LLaVA: Apr 17, MiniGPT-4: Apr 20, and LLaMA Adapter: Mar 28) appeared online within two months of a submission (May 18). According to NeurIPS 2023 policy, they are generally considered \"contemporaneous\".\nIt is beyond our ability to include these vision-language models in submission. \n\nThanks for your advice to enhance evaluation baselines, which we will add in our final version. We show an additional experiment here, which is based on a more recent vision-language model Otter [1] (May 5) (OTTER-9B-INIT) similar to LLaVA, MiniGPT-4, and LLaMA Adapter. The results are shown below.\n\n| Method      |  Stanford40 zero-shot mAP |\n| ----------- | ----------- |\n| Otter[1]             | 44.83        | \n| Otter[1]+Reasoning   | 47.16        | \n\nThus, the effectiveness of our method is verified on the SOTA System-1-like method Otter [1].\nNote that, we find the performance of Otter [1] on Stanford40 zero-shot is relatively low compared with CLIP and BLIP2. A possible reason is, the model tuned with visual instructions targets captioning tasks and requires more advanced transitions to solve classification tasks.\nDuring the time-starved rebuttal period, we try our best to sketch this additional experiment, and a more detailed investigation is left for future work.\n\n[1] Li, Bo, et al. \"Otter: A multi-modal model with in-context instruction tuning.\" arXiv preprint arXiv:2305.03726 (2023).\n\n## Writing\nThanks for your advice, we will revise the final version:\n\n- add clearer explanations\n- make sections 3.1-3.2 more concise\n- detail method designs in the early part of the paper\n- add related references\n"}, {"review_id": "KZFcbTZK1m", "reviewer": "Reviewer_SHNq", "summary": "The paper proposes to increase coverage and generalization of logic-based methods to model 'system-2' or deliberative reasoning for the task of human activity prediction in images. Specifically, the authors incorporate an LLM (GPT3.5) to obtain a larger number of symbols and inference rules related to human activity prediction and further also utilize an LLM to check logical entailment of generated rules (based on which noisy rules are filtered). Their proposed symbolic system operates upon a VLM which is used for symbol extraction from the image, based on which rules are activated with a fuzzy-logic computation (to obtain a probability score for the inference chain). \n\nAuthors show that adding their method on top of two VLMS (CLIP and BLIP2), enhances their performance on prominent activity recognition benchmarks and also show visualizations of activated rules and predictions besides ablation studies of their method.", "questions": "Please see weaknesses. \n\nMy primary concerns (impacting rating) are regarding:\n1. Does compute scale linearly with number of symbols and would this be a major limitation in practice? If so, can this can be addressed through alternative symbol grounding/identification methods? \n2. The relatively limited evaluation compared to other relevant works for human activity prediction. How does the method compare to related baselines in human-object interaction learning such as ICompass (Interaction Compass: Multi-Label Zero-Shot Learning of Human-Object Interactions via Spatial Relations (ICCV 2021) and RLIP (NeurIPS 2022)) or alternatively multimodal LLM methods (e.g. PICa in \"An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA\" AAAI 2022)?", "limitations": "Potential negative societal impact is mentioned to be minimal.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The usage of LLMs to broaden semantic coverage of symbolic/logic-based methods by discovering relevant premises and rules and also for checking/approximating logical entailment is relatively novel. \n\n2. The methodology and formalism of their approach is well described, with appropriate examples and figures/tables to help reader understand the approach.\n\n3. Results on multiple prominent benchmarks (including HAKE, HICO and Stanford-40) for human activity prediction show that the method can benefit both zero-shot (for BLIP2 and CLIP) and finetuned (for CLIP) performances of prominent VLMs. Ablation results also highlight contributions of individual method components/variations.", "weaknesses": "1. The symbolic system's application draws on using a VLM multiple times to extract the probability of individual symbols. It thus seems that the computation will linearly scale with number of symbols, which can be a significant drawback due to the incurred computation cost.  (VLMs such as BLIP2 by themselves have very large computation cost, hence even increasing that 100x for 100 symbols is a very significant additional compute). Further, extending this to further domains beyond human-activity recognition would require many more symbols, and ultimately possibly not be scalable. \n\n2. Evaluation baselines only consider results of the backbone VLMs currently. There are multiple recent works in related direction of human-object interaction and activity prediction in zero- or few-shot scenarios which have not been mentioned or used as baselines (e.g. Weakly-Supervised HOI detection (ICLR 2023); RLIP (NeurIPS 2022)). Currently only RelVIT is mentioned in supplemental. Authors should either compare with such methods or mention why these recent methods are not used/suitable for comparison. \n\n3. Further, since LLMs are employed for much of the 'system-2' reasoning, authors should consider reporting results of alternative methods using VLMs + LLMs directly without any logical system (e.g. VLM generates image caption, based on which LLM predicts possible activity when prompted for each activity in the label space -- similar to PICa in  \"An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA\" AAAI 2022). These results could better highlight the contribution of the proposed logical system and that performance increments are not due to just LLM-VLM combination.\n\nLess important:\n1. Failure cases could be illustrated for a small sample of the questions to better highlight which component is currently leading to errors (e.g. is it the VLM not producing accurate symbols or is it a noisy rule, etc)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n## Computation Scalability\n\nSymbol predictions will increase the computational cost as a trade-off for explainability and generalization. However, with the proper method design shown below, the computational cost can be largely reduced and will not be a major limitation.\n\n1. Not all symbols need to be predicted via pruning. Symbols can be organized as a tree with hierarchical prediction. \nFor the Fig.6 example, $m_1$...$m_{10}$ can be organized as:\n    - $m_a$: Interact with a seller:\n        - $m_1$:Talk with seller\n        - $m_3$: Seller hand over orange\n        - $m_{10}$: Give money to seller\n    - $m_b$: Interact with the oranges:\n        - $m_2$: Reach for an orange\n    - $m_c$: Interact with a container:\n        - $m_5$: Place orange in a bag\n        - $m_6$: Pick orange from a basket\n        - $m_7$: Hold a bag of oranges\n        - $m_9$: Seller put the orange in bag\n    - $m_d$: Payment Process:\n        - $m_4$: Stand in front of fruit stand\n        - $m_8$: Reach for a wallet\n\nThe father symbols (e.g., $m_a$) \ncan be summarized based on the son symbols (e.g., $m_1$) via simple semantic extraction (e.g., LLM prompt).\nOn the symbol tree, the probability of a son symbol is no higher than its father symbol. \nWhen the probability of a father symbol is lower than a threshold (e.g., in image III, $m_a$ probability less than 0.1), its son symbols ($m_1$, $m_3$, $m_{10}$) will be assigned the threshold directly without extra computation. \n\nThus, the computational cost is saved via the hierarchical structure of symbols. For example, image III only needs 5 instead of 10 calculations: Interact with a seller(x), Interact with the oranges, Reach for an orange, Interact with a container(x), and Payment Process(x). The computation save will be more evident with the number of symbols scales.\n\n2. Activity prediction can be determined with only part of known symbols, which benefits from the fuzzy-logic calculation with the simple minimum/maximum function.\nTake the image I in Fig.6 as an example. When the probabilities of $m_1=0.92$, $m_2=0.14$, $m_3=0.94$, $m_4=0.57$ are known, we have $p(r_1)=0.14$, $p(r_2)=0.92$, $p(r_3),p(r_4),p(r_5)\\leq0.57$, thus determining the activity prediction $p(c)=0.92$ with only 4 known symbols.\n3. Different activities have different complexity and computational cost. \n    - For those simple activities (e.g., \"hold an apple\"), they have few rules or can even be predicted without reasoning. Thus, the computational cost is controllable.\n    - Our reasoning is especially beneficial to those complex activities (e.g., \"board an airplane\") with various patterns and insufficient training samples. Thus, the computational cost proposed symbolic system is a trade-off to compensate for the data insufficiency and solve complex activities.\n\n## VLMs+LLMs Ablation\nTo highlight the contribution of the proposed logical system, we conduct experiments where VLMs and LLMs are combined in different ways. Results on zero-shot Stanford-40 are shown below. \n\n1. LLM only provides implicit reasoning for VLM (BLIP2): 91.85 mAP\n2. LLM is equipped with rules, and the rules are prompted in the label space\n    - Prompt LLM in the VLM. The text prompt of BLIP2 contains not only the target activity but also its rules generated from our symbolic system: 91.97 mAP\n    - Prompt an external LLM like PICa. BLIP2 generates image captions, and an LLM make activity prediction based on the caption and the rule prompts: 73.18 mAP\n3. LLM is equipped with rules and explicit reasoning (ours): 92.59 mAP\n\nWe can find that the proposed explicit reasoning is superior to using rules as LLM prompts because of the precise knowledge representation. When prompting rules in LLM's label space, inserting the prompt in a VLM is superior. Using an external LLM fails possibly because the image caption fails to capture the accurate semantics for LLM to judge. However, there are more designs left to explore, e.g., in-context learning in PICa.\n\n## HOI Learning Baselines\n\nWe respectfully explain that we focus the reasoning methodology on *image-level* activity understanding benchmarks. Thus, only methods (e.g., RelViT) on HOI recognition benchmark HICO are listed as baselines, and HOI detection methods (e.g., Weakly-HOI, RLIP) on HICO-DET are not listed. Since HOI detection requires additional designs for detecting human-object pairs, we leave it to future work.\nInstead, we use several other image-level activity understanding benchmarks, e.g., Stanford40 (action recognition), and HAKE-PaSta (conditional PaSta Q-A). Our experiment settings are more similar to RelViT (focus on reasoning on HICO, GQA dataset) than to HOI detection methods (focus on HICO-DET, VCOCO dataset).\n\nAdditionally,  ICompass shows zero-shot HOI recognition performance in Tab. 1, where some HOI categories are seen and others are unseen. Our method focuses on reasoning while ICompass focuses on action/object score composition. Thus, we currently use it simply as a baseline and will add it in the updated version.\n\n\n## Failure Cases Analysis\n\nWe provide two failure cases (false negative) for the activity \"human board an airplane\" in the attached pdf Fig.2. \nIn image I, the failure is caused by the prediction error of symbol probabilities (which we name \"System-1 error\"). Some symbols are predicted as low probabilities by mistake, e.g., \"walk towards the boarding gate\". The symbol prediction error is possibly caused by tiny human bounding boxes.\nIn image II, the failure emerges because the generated rules do not cover the situation in the image, i.e., \"System-2 defects\". For example, the image contains the symbol \"climb up the airplane edge\" which is not in the symbolic system.\n\nWe also make quantitative analysis on System-1 errors and System-2 defects, which is detailed in the response for Reviewer zui3 \"System-1/2 Ablation\"."}, {"review_id": "PQngcjCHWb", "reviewer": "Reviewer_K3QA", "summary": "The paper addresses the problem of visual activity understanding and proposes a way to leverage large visual and language models to i) propose symbols from an image, ii) derive new symbols via logic entailment and perform classification. The results show that the augmented system with symbolic reasoning capabilities performs slightly better than the equivalent large models without the reasoning system. ", "questions": "- How is the set of the initial symbols selected? Isn't this initial set generation prone to the same issues as some of the related works? \n- Given the form of each rule having multiple premises and a single conclusion, why is the hyper-graph structure needed? Wouldn't a simple graph with multiple directed edge be enough?\n- ", "limitations": "The core of the proposed work is based on utilising pre-trained large models which require vast amounts of data and computational resources and so the proposed method is as efficient as described only by assuming access to such large models.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper proposes an interesting way to leverage large visual and language models to derive a set of symbols relevant to a particular picture and the action being executed in it. Constructing an ontology based on the compiled knowledge represented in an LLM seems to be a powerful approach to enable task specific higher level reasoning.", "weaknesses": "One of the key improvements over other related work is claimed to be the broad semantic coverage of the derived symbols and the generated rules. However, given the statistical nature of the used large models it is not clear how accurate these symbols and rules actually are. A deeper analysis of the generated ontology is definitely needed in order to assess the applicability of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback. We will add more discussions about limitations brought by using pre-trained large models. We hope to address the questions and concerns below.\n\n## Detailed Evaluation\\&Analysis\n\nWe respectfully explain that using LLM is the optimal method within our consideration, which balances abundant knowledge and low manual cost. More reliable knowledge than LLM should be extracted from human annotations, but complete human evaluation of generated symbols \\& rules is quite expensive. \nTherefore, for deeper analysis, we currently focus on a small but representative subset. On this test subset, we verify whether the proposed symbolic system indeed brings advantages (e.g., broader semantic coverage, and more rational rules to avoid confusion). Also, we analyze the performance bottleneck of System-1 errors and System-2 defects. Furthermore, we analyze and evaluate the robustness of the generated ontology. \nDetails are shown below. We will scale the experiment and detail it in our final version. \n\n### The test set\nWe select a small subset of the HICO test set (120 images for 10 activity classes). The 10 activity classes are: \"human board/wash an airplane\", \"human park/repair a bicycle\", \"human feed/race a horse\", \"human break/sign a baseball bat\", and \"human buy/wash an orange\". These activities have relatively complex visual patterns and cover different interacted objects.\nFor each image and each of its ground-truth activities, we list all of the symbols related to the activity in the symbolic system and annotate whether a symbol happens (0/1) in the image via human judgment.\n\n### Symbols\\&Rules Evaluation\nThe proposed symbolic system has broader semantic coverage than HAKE. For quantitative measurement, we count the happening symbols for each image-activity pair under different symbolic systems. The average number is 4.6 for ours and 1.8 for HAKE. Though a rough estimation, the gap in symbol counts indicates that the symbol semantic of HAKE is limited. Furthermore, in our symbolic system, different activities have different symbols, further boosting semantic coverage.\n\nThe proposed symbolic system avoids confusion with more rational rules.\nWe evaluate confusion by counting different image-activity pairs which share the same symbols. For HAKE symbolic system, the confusion problem is severe, with 261 confusion pairs over $C_{120}^2=7140$ pairs (3.7\\%). For our symbolic system, there are no confusion pairs on the test set because of the presentation ability of symbols and entailment check. \n\n### Bottleneck Analysis\nWe investigate the impact of System-1/2 separately as below. \nWith a perfect System-1, we assume the probabilities of symbols are definitely known as 0/1 instead of $p \\in [0,1]$. The ground-truth symbols are annotated as explained above. \nWith a perfect System-2, we assume that the generated rules cover all samples (image-activity pairs). \n\n| System-1    |  System-2   | mAP        |\n| ----------- | ----------- |----------- |\n| perfect     | perfect     | 100.00     |\n| perfect     | imperfect   | 89.13      |\n| imperfect   | perfect     | 80.07      |\n| imperfect   | imperfect   | 72.59      |\n\nThe performance drop [100.00$\\rightarrow$89.13 mAP, 80.07$\\rightarrow$72.59 mAP] indicates the System-2 defects, i.e., the generated rules have not covered all samples (image-activity pairs). The drop [100.00 $\\rightarrow$ 80.07 mAP, 89.13 $\\rightarrow$72.59 mAP] indicates the System-1 errors, i.e., symbol probabilities cannot be predicted accurately. On this test set, System-1 errors have more negative impacts on the final performance.\n\n### Robustness Evaluation\nConvergence. The symbol-rule loop should find a symbolic sub-system that \"converges\". In other words, after a certain stopping criterion, a new round will bring few or even no new symbols to the sub-system. \nIn practice, the stopping criterion is: At most 15 symbols are used for rule extension to avoid semantic redundancy (L209).\nWith this stopping criterion, the symbol-rule loop satisfies convergence. For the randomly sampled 50 activities, a new round brings an average of 1.9 symbols to the sub-system, verifying the convergence.\n\nLow sensitivity to initial conditions. The symbol-rule loop should provide a reliable and stable symbolic sub-system over a range of initial conditions. \nWe randomly sample 15 different initial symbol sets to generate the sub-system and then measure the average pairwise graph similarity. The average similarity (range: [-1,1]) on 10 randomly sampled activities is 0.91, verifying the low sensitivity to initial conditions.\n\n## Method Details Explanation\n### Initial Symbols Selection\n\nThe initial symbols are five hand-related states related to the activity, which are generated by prompting an LLM (L147, appendix L78-81).\n\nThe initial symbols are simply used as triggers for rule generation, while the rule extension is the main component to extract LLM knowledge and guarantee broad semantic coverage. Fig. 4 and Tab. 1 show an example where initial symbols can expand to a satisfying symbolic sub-system via symbol-rule loop and entailment check. Furthermore, the symbol-rule loop has low sensitivity to initial symbols (see above). Thus, the initial symbols have a relatively low impact on a symbolic sub-system.\n\n### Hyper-graph Structure\nIn the symbolic system, there are two \"multiple\" relationships: each activity has multiple rules, and each rule has multiple premises. We find hyper-graph a proper structure to clarify it, where one rule is a hyperedge with connecting multiple vertices (premises). Maybe other alternative structures can also be used to clarify the system.\n"}, {"review_id": "YbGS93Ilga", "reviewer": "Reviewer_zui3", "summary": "In this paper, a symbolic-System-2 submodule is integrated with a System-1 module to perform human activity recognition. The work follows the intuitive idea, where a visual prediction module extracts visual symbols and a symbolic reasoning module performs logical reasoning to recognize activities. For the visual symbol extraction module, the work applies a general VLM, to predict the truth value of a predicate in a logic clause. The logical system then performs fuzzy logic to predict the final outcome of the rule, ie, conclusion. The logical system is constructed from LLM, without expensive human annotation in existing works. In experiments, it is shown that the model is better than existing baselines.", "questions": "See above.", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ The work leverages and extends the intuitive idea of System-1 and System-2 combination. The integration is important compared to System-1 only pure predictive framework and it's really encouraging to see more and more works integrating these two systems in typical vision tasks.\n+ Compared to existing works, the method leverages LLM for knowledge-base extraction. While the cost saving is from LLM, rather than some nice problem structure design, I do appreciate the efforts to construct formal logic base from natural language systems and make it work.\n", "weaknesses": "I'm not an expert in logical system design so I will refrain from commenting on the system design part on logics. However, I do see some general problems regarding this work.\n\nFor one, designing logical system specifically for a task risks overfitting. While the performance is improved on a specific task, I do note that the slight increase may not be worth the generality from models like CLIP. Besides, the logic system is pretty complicated. And I do not think the system, or the system construction method could be easily adapted for other tasks. On the other hand, such a system is also only demonstrated in a single task. I doubt if the method could be more general. \n\nI'm also interested in seeing more detailed ablation study. In particular, whether it is the System-1 part of the System-2 part that more adversely impacts performance. Would it be possible to assume perfect System-1 and test the System-2 performance or vise versa? Besides, while in general the idea should work. There are many subtleties in telling the truth value of statements, eg, the leg is ``close'' to something. How should such subtleties be addressed? \n\nThose being said, I admit the inherent conflict between logical system that are delicate, explainable, robust, and generalizable and deep learning system that are general but fragile. I do not see these points as weaknesses but rather interested in hearing how the authors see them.\n\nFor my final rating, I'd like to sync with fellow reviewers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n## Generality\nThe method could be more general. We choose activity understanding as a good and important initial test bed because it is a difficult task with complex visual patterns and compositional nature. Based on the contribution of this paper, we step towards solving the integration of System-1/2 for general visual understanding.  The proposed symbolic system can be generalized to various tasks (e.g., classification, visual commonsense) with a similar formulation. Some examples are shown below:\n\n1. Object classification\n    - For the object \"crocodile\" in CIFAR-100 [1] dataset, one example rule is: long, slightly curved body $\\wedge$ four short legs $\\wedge$ ... $\\wedge$ a long, muscular tail $\\rightarrow$ crocodile\n\n2. Visual commonsense\n    - For one example QA in VCR [2] dataset (shown in attached pdf Fig.1), one example rule is: person on the right shows affectionate gestures $\\wedge$ person on the left engages in conversation $\\wedge$ ... $\\wedge$ The environment is a restaurant $\\rightarrow$ b\n    \nUnder this formulation, semantic coverage and logical entailment remain two important properties under a more general setting, and we can reason with visual inputs via a similar pipeline. \nWe conduct experiments with two zero-shot benchmarks: \n\n1. CIFAR-100 [1] test set.\n\n| Method           | CIFAR-100 Acc (\\%) |\n| -----------      |  -----------        |\n| CLIP             | 77.50               |\n| CLIP+Reasoning   | 78.31               |\n\n\n2. VCR [2] val set. BLIP2 is adopted as a baseline as it is a VQA-style task. VCR val set has 26534 image-question pairs, where we select a subset of 557 pairs for the test. The selected pairs are highly related to human activities (instead of role, mental, .etc) and only involve *one* person's activity. \n\n| Method            | VCR Acc (\\%)|\n| -----------       | ----------- |\n| BLIP2             | 46.32        | \n| BLIP2+Reasoning   | 47.08        | \n\nWe can see that the proposed pipeline can be applied to general visual tasks and facilitate reasoning. In this paper, we focus on method design instead of task generalization.\nWe will add these explanations in the final version and analyze more general settings in future work.\n\n[1] Krizhevsky, Alex, and Geoffrey Hinton. \"Learning multiple layers of features from tiny images.\" (2009): 7.\n\n[2] Zellers, Rowan, et al. \"From recognition to cognition: Visual commonsense reasoning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.\n\n## System-1/2 Ablation\n\nThanks for your advice. We conduct a detailed ablation study on a small subset of the HICO test set (120 images for 10 activity classes). We investigate the impact of System-1/2 separately. \nWith a perfect System-1, we assume the probabilities of symbols are definitely known as 0/1 instead of $p \\in [0,1]$. With a perfect System-2, we assume that the generated rules cover all samples (image-activity pairs). To construct a perfect System-1, for each image and each of its ground-truth activities, we list all of the symbols related to the activity in the symbolic system and annotate whether a symbol happens (0/1) in the image via human judgment. The results are shown below.\n\n| System-1    |  System-2   | mAP        |\n| ----------- | ----------- |----------- |\n| perfect     | perfect     | 100.00     |\n| perfect     | imperfect   | 89.13      |\n| imperfect   | perfect     | 80.07      |\n| imperfect   | imperfect   | 72.59      |\n\nThe performance drop [100.00$\\rightarrow$89.13 mAP, 80.07$\\rightarrow$72.59 mAP] indicates the System-2 defects, i.e., the generated rules have not covered all samples (image-activity pairs). The drop [100.00 $\\rightarrow$ 80.07 mAP, 89.13 $\\rightarrow$72.59 mAP] indicates the System-1 errors, i.e., symbol probabilities cannot be predicted accurately. On this test set, System-1 errors have more negative impacts on the final performance.\nWe will scale the experiment and detail it in our final version. \n\n## Symbol Recognition Subtleties\n\nThough some complex symbols (e.g., the leg is \"close\" to something) might be difficult to recognize, such subtleties do not hinder the main benefit brought by the proposed symbolic system and reasoning pipeline. The analysis is detailed below:\n\n1. Visual symbols are passed into a checked module, where those with uncertain predictions are filtered out (main paper L183-186, appendix L121-125). Thus, complex symbols which are difficult to recognize are excluded from the reasoning process instead of misleading reasoning.\n2. After filtering out uncertain symbols, the remaining, easily-recognized symbols are informative enough to facilitate activity reasoning, which is verified via experiments (main paper Tab. 2) and visualization examples (main paper Fig. 6).\n3. Instead of being directly discarded, some complex symbols can also be further decomposed into more simple symbols. Then the decomposed simple symbols can facilitate reasoning. For example, \"wave goodbye to loved ones\" (an example by Reviewer yZAX) can be decomposed into \"arms lift\", \"hand swing\", and \"eyes look at loved ones\". We leave it to future work."}, {"review_id": "87XELtFiG9", "reviewer": "Reviewer_yZAX", "summary": "This paper studies the use of a symbolic system for understanding human activity. The author argues that a good symbolic system for human activity understanding should possess a diverse range of symbols that accurately identify the human activity, and the rules within the symbolic system should be logically correct. The paper proposes the utilization of LLMs to guarantee these two properties and suggests building a comprehensive pipeline incorporating both LLMs and VLMs. Experiments demonstrate that the proposed system outperforms the baseline model, which solely employs VLMs, on multiple datasets.", "questions": "See my comments in the weaknesses section.", "limitations": "The authors have a paragraph discussing the limitations of the proposed method in the last section.", "rating": 5, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written and easy to follow.\n- The paper identifies two ideal properties of a good symbolic system: semantic coverage and logical entailment, which is very reasonable.\n- Utilizing the commonsense knowledge of LLMs to achieve these two properties is also promising.\nThe experiments have demonstrated the effectiveness of the model on multiple datasets.", "weaknesses": "- Some crucial details of the method are still missing:\n    - Regarding symbol initialization, it is not explicitly mentioned how the prompt in Appendix 3.3 (Line 79) is generated. It would be helpful to clarify whether it is manually written or generated using a template, and if so, what is the template.\n    - There are two versions of prompts used for rule extension in the main paper (Line 139) and the appendix (Line 84). In the main paper, a single known symbol is used, while in the appendix, all known symbols are used in the prompt. Additionally, in the appendix, the first sentence contains specific information related to the airplane case: \"In a picture, **there is an airplane**\". How this particular prompt is generated?\n    - According to Algorithm 1, symbols that contribute to a rule passing the entailment check should be removed from the candidate symbol list (Line 13). However, in Figure 4, after the 1→6→7→8 rule is generated, there are still rules such as 2→1 and 3→1→9 being generated. It would be useful to provide an explanation for this discrepancy.\n- According to Table 1, some generated symbols are more complex than the target activity, such as \"wave goodbye to loved ones,\" and some do not follow the format of \"<The person's hands/arms/hip/legs/feet> <verb> <object>\", like \"airline staff checking the boarding pass\". Given the complexity of the generated symbols, VLMs might face challenges in recognizing them in the images. Although experiments have shown that the current pipeline performs better than directly using VLMs on multiple datasets, there is no guarantee that VLMs can easily recognize the generated symbols.\n- While the author proposes two ideal properties that should be satisfied, the experiments only focus on the final accuracy of the models on human activity understanding tasks. The accuracy heavily relies on the performance of the VLMs employed. There is no evaluation of how well LLMs ensure these two properties. This makes it unclear what the contribution and bottleneck of the proposed system are. One potential experiment the authors could consider is to have a subset of the dataset and evaluate the pipeline with VLMs replaced by human judgment. But other experiments are also appreciated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n### Method Details Explanation\n1. Symbol Initialization\n    - The prompt is manually designed by us. Special constraints are considered in the prompt to output expected initial prompts. For example, we constrain the answer symbols as hand-related states since they are more common than other symbols (e.g., feet-related states, scenes). The optimal prompt would be found if the prompt can be generated automatically. We leave it to future work.\n\n2. Rule Extension\n     - The rule extension prompt in the main paper L139 is a brief version, while the prompt in appendix L84 is a more detailed one.\nFor the main paper version, \"IF [<known symbol>]\" refers to listing all the known symbols, which is detailed in the appendix version.\nFor human-object interaction activity where an interacted object is known, it is listed in the prompt (e.g., \"there is an airplane\"), which is omitted in the main paper version. We will add explanations in the final version.\n\n3. Fig-4 Explanation\n    - We respectfully explain that there is a misunderstanding of Algorithm 1 Line 13. After the 1→6→7→8 rule is generated, the initial symbol 3 is extended into a rule with symbol 1,9 added. According to Line 13, since symbol 1 already exists in $S_c$, it is not pushed into the candidate symbol list to generate the rule 1→6→7→8 repeatedly, and only symbol 9 is added as a new candidate symbol. \nIt does not mean that rule 3→1→9 will be removed. Symbol 1 can be reused because it is normal for rules to share some symbols.\n\n### Symbol Recognition\n\nThough some complex symbols might be difficult to recognize, it does not hinder the main benefit brought by the proposed symbolic system and reasoning pipeline. The analysis is detailed below:\n\n1. Visual symbols are passed into a checked module, where those with uncertain predictions are filtered out (main paper L183-186, appendix L121-125). Thus, complex symbols which are difficult to recognize are excluded from the reasoning process instead of misleading reasoning.\n2. After filtering out uncertain symbols, the remaining, easily-recognized symbols are informative enough to facilitate activity reasoning, which is verified via experiments (main paper Tab.2) and visualization examples (main paper Fig. 6).\n3. Instead of being directly discarded, complex symbols can also be further decomposed into more simple symbols. Then the decomposed simple symbols can facilitate reasoning. For example, \"wave goodbye to loved ones\" can be decomposed into \"arms lift\"\"hand swing\" and \"eyes look at loved ones\". We leave it to future work.\n\n### Contribution\\&Bottleneck Evaluation\n\nThanks for your advice. We conduct additional evaluations for deeper insight into the results. Currently, the experiment is conducted on a small subset of the HICO test set (120 images for 10 activity classes).\nFor each image and each of its ground-truth activities, we list all of the symbols related to the activity in the symbolic system and annotate whether a symbol happens (0/1) in the image via human judgment.\nConclusions, experiments, and analyses are detailed below. We will scale the experiment and detail it in our final version. \n\n1. The proposed symbolic system has broader semantic coverage than HAKE. \n    - For quantitative measurement, we count the happening symbols for each image-activity pair under different symbolic systems. The average number is 4.6 for ours and 1.8 for HAKE. Though a rough estimation, the gap in symbol counts indicates that the symbol semantic of HAKE is limited.\nFurthermore, in our symbolic system, different activities have different symbols, further boosting semantic coverage.\n2. The proposed symbolic system avoids confusion with more rational rules.\n    - We evaluate confusion by counting different image-activity pairs which share the same symbols. For HAKE symbolic system, the confusion problem is severe, with 261 confusion pairs over $C_{120}^2=7140$ pairs (accounting for 3.7\\%). For our symbolic system, there are no confusion pairs on the test set because of the presentation ability of symbols and entailment check. \n3. Reasoning is bottlenecked by System-1 errors (more) and System-2 defects (less).\n    - We investigate the impact of System-1/2 separately. With a perfect System-1, we assume the probabilities of symbols are definitely known as 0/1 instead of $p \\in [0,1]$. The ground-truth symbols are annotated as explained above. With a perfect System-2, we assume that the generated rules cover all samples (image-activity pairs). The results are shown below.\n\n| System-1    |  System-2   | mAP        |\n| ----------- | ----------- |----------- |\n| perfect     | perfect     | 100.00     |\n| perfect     | imperfect   | 89.13      |\n| imperfect   | perfect     | 80.07      |\n| imperfect   | imperfect   | 72.59      |\n\nThe performance drop [100.00$\\rightarrow$89.13 mAP, 80.07$\\rightarrow$72.59 mAP] indicates the System-2 defects, i.e., the generated rules have not covered all samples (image-activity pairs). The drop [100.00 $\\rightarrow$ 80.07 mAP, 89.13 $\\rightarrow$72.59 mAP] indicates the System-1 errors, i.e., symbol probabilities cannot be predicted accurately. On this test set, System-1 errors have more negative impacts on the final performance.\n"}, {"review_id": "GSTLSoyptd", "reviewer": "Reviewer_sPA9", "summary": "This work is in the area of human visual activity reasoning and tries to improve System-2 (logic, deliberative reasoning) integrated with System-1 (that deals with intuitivity, associativity, and correlation) to improve explainability, generalization and efficiency which is demonstrated through empirical evaluation. This work attempts to improve over existing methods which use limited symbols, and are therefore is insufficient to cover complex patterns of activities for visual activity understanding. They do this by proposing a new system with broad-coverage symbols and rational rules.", "questions": "Most questions given above in Weaknesses.\n\nFrom Fig 6, it is not clear if the method will generate the correct set of rules if the images also contained say apples in various circumstances in the background, but the person only decides to buy orange.  Can you please clarify this?", "limitations": "One limitation regarding symbolic representations of continuous activities is mentioned.  The authors have not addressed negative societal biases, and I can think of cases where some societal biases might arise.  Activity-based reasoning is at its core a kind of \"stereotype\"-based reasoning: \"Balls are round.  Balls roll.  Apples are usually round.  Apples probably roll.\" In the domain of common objects, the risks of such stereotype-based reasoning are probably minimal, although if other objects with potentially harmful affordances (e.g., firearms or other weapons) are used without noting that those behaviors are harmful and should be avoided, this presents potential for abuse.  Can things like race, gender, or social class be represented symbolically? Should they?  Would symbolic representation of these bias system outputs in the presence of \"coded\" objects like firearms?  Beyond universal basic behaviors, who decides what the symbolic representations of them are? Without attending to these questions and more, there is a risk of attributing \"typical\" or \"stereotypical\" behaviors to certain individuals or groups of people, and risk generalizing them to other individuals, and hence reproducing patterns of bias and discrimination.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "-This work attempts to solve the lack of compositional generalization (generalized understanding of visual concepts, their relationships, and novel combinations) by creating a system of symbols with broad semantic coverage and rules that are rational and unambiguous. E.g., an existing work contains the following rule:\nHand hold ^ Feet close with |= human ride a boat. \nThis rule does not cover the complex patterns of human activities.\n\n- This work improves the existing methods by using an inexpensive annotation method by utilizing LLMs (OpenAI GPT3.5 API). This method is able to generate diverse symbols and rules quickly as shown in their symbolic system experiment. For the semantic coverage of the huge image dataset (the target domain), they replace it (the target domain) with the semantic coverage of language models.", "weaknesses": "The core message of the paper appears to be that more symbols with greater coverage leads to improved performance.  Figure 4 provides a high-level view of how these symbols are generated from LLM output, and Figure 6 notionally shows how they integrate with visual processing, but technical details are missing or unclear. For instance, how are initial symbols \"extended into rules satisfying entailment\"?  In what format are symbols and rules stored?  What is the method for integrating the symbolic reasoning system into the overall visual processing pipeline?\n\nTheir method uses symbol-rule loop to generate new rules whereby they use a known symbol to generate a new symbol and connect it with the rules. The author(s) do not present examples to test the robustness of this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n### Method Details Explanation\n\nThanks for your advice. We will add more explanations in the final version.\n\n1. Rule Extension\n    - It is explained in the main paper Sec. 3.2.3 and Alg. 1 (L6-10). The initial symbols are extended into rules by adding one new symbol and checking entailment in an interleaved way.\n\n2. Format of Symbols\\&Rules\n    - Symbols are stored as natural language descriptions, and rules are stored as a premise symbol set and a conclusion symbol.\nIt is explained in the rule examples (L45) and the symbolic system definition (Def.3).\n\n3. Visual Reasoning\n    - It is explained in Sec. 3.3 and Fig. 3, where the activity prediction is calculated based on the rules of the symbolic system and the probabilities of the symbols.\n\n4. Rule in Fig. 6\n    - We analyze whether the method will make errors when there are contexts (apple) disturbing the activity prediction (human buy orange) in the image. \n    - The unrelated contexts are less likely to hinder the accuracy of the generated rules because the entailment check guarantees the rationality of the rule. For the activity \"human buy orange\", there are symbols explicitly pointing out the existence of \"oranges\" and the interaction with \"oranges\" instead of \"apples\" (e.g., $m_7$: hold a bag of oranges). \nWhen there are apples in the background, the reasoning will focus on whether the person is really \"holding\" a bag of oranges.\nTherefore, the correct set of rules is generated.\n\n\n### Symbol-Rule Loop Robustness\n\nFig. 4 and Tab. 1 present an example of rule generation loops and reveal some phenomena, e.g., repeated rules, and early stop of rule extension.\nWe further analyze robustness as follows. Additional analysis and results will be updated.\n\n1. Convergence. The symbol-rule loop should find a symbolic sub-system that \"converges\". In other words, after a certain stopping criterion, a new round will bring few or even no new symbols to the sub-system. \n    - In practice, the stopping criterion is: At most 15 symbols are used for rule extension to avoid semantic redundancy (L209). After that, new symbols are not used for rule extension, leading to a stop of the loop.\n    - With the stopping criterion, the symbol-rule loop satisfies convergence. Take Tab.1 as an example: in rounds 1-5, the newly added symbols count is 5, 5, 3, 3, 1, which shows a convergence trend. Fig.5(a) shows a similar convergence trend. More generally, for the randomly sampled 50 activities, a new round brings an average of 1.9 symbols to the sub-system, verifying the convergence.\n2. Low sensitivity to initial conditions. The symbol-rule loop should provide a reliable and stable symbolic sub-system over a range of initial conditions. \n    - The sensitivity to initial conditions mainly comes from the asymmetry of rule generation: with 4 as an initial symbol, a rule connecting 4\\&10\\&6 is generated, while the rule sometimes cannot be generated with 10 as an initial symbol (L214). \n    - However, we find such sensitivity an infrequent case with multiple rules sampling (L215-216). To verify this, we randomly sample 15 different initial symbol sets to generate the sub-system and then measure the average pairwise graph similarity. The average similarity (range: [-1,1]) on 10 randomly sampled activities is 0.91, verifying the low sensitivity to initial conditions.\n\n### Limitation Discussion\n\nThanks for your advice. We will add discussions about the bias and stereotypes brought by symbolic computation in the limitation section.\n"}, {"review_id": "gn4PwaqsLf", "reviewer": "Reviewer_yAvu", "summary": "The paper addresses  activity reasoning, where the task is to detect human activities given visual inputs. \nTo achiever compositional generalization for activity understanding, the paper propose a new symbolic system that is capable of relational reasoning.\nIn the experiments, the proposed system improved naive CLIP models showing its strong reasoning capability.", "questions": "- The rule is denoted using the logical entailment $\\models$. Is this different from implication $\\rightarrow$, which is commonly used for logic?  (In Tab. 2 in the appendix, $\\rightarrow$ is used.)\n- How the proposed symbolic system is related to other symbolic reasoners, e.g. standard first-order logic? What are the pros and cons?", "limitations": "The authors listed some limitations in the paper.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, the paper addresses an important problem for integrating symbolic reasoning with neural perception models.  The paper demonstrates an instantiation for such an integration by providing formal definitions.  The paper conveys its idea with a good presentation using clear visualizations. The obtained results are interesting. \nThese contributions are beneficial for the community to build agents that can perform complex reasoning and understand the world better beyond simple visual perception.", "weaknesses": "My major concerns regarding this paper are listed as follows.\n- The notation and semantics of the proposed method are unclear.  The rule is denoted using the logical entailment $\\models$. Is this different from implication $\\rightarrow$, which is commonly used for logic?  In Tab. 2 in the appendix, $\\rightarrow$ is used.  Since the definition of logical entailment is not provided in the paper, it is unclear how the rules can be understood as formulae. The implication and entailment articulate different concepts.\n- Related to the first point, it is not clear how the proposed symbolic system is related to other symbolic reasoners, e.g. standard first-order logic? What are the pros and cons?\n- The paper mainly addresses activity reasoning, but it is not well-described in the paper. Even though some visualizations are provided in the paper,  it is not easy to follow the arguments for readers who are not familiar with this topic. It would be beneficial to provide some simple examples throughout the paper.\n- Related to the previous point, the empirical setting is not easy to follow. E.g. on line 232 , it says, \"We follow the setting of HAKE [20] to integrate the reasoning ...\", however, this explanation is not self-contained. The experimental settings should be articulated more clearer in the paper.\n\n- Other minor issues:\n\t- Definition 5 is not clear. What are activity semantic and ongoing symbols? ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the feedback and hope to address the questions and concerns below.\n\n### Logic Notations\nThe notations \"$\\models$\" (semantic entailment) and \"$\\rightarrow$\" (syntactic entailment) both describe the concept of one statement leading to another, but their emphasis is different.\n\nSemantic entailment focuses on truth tables or interpretations (i.e., under what circumstances a proposition is true), while syntactic entailment focuses on proof processes based on formal reasoning principles.\nVisual reasoning focuses on interpretations instead of syntactic proof. Thus we replace the notation \"$\\rightarrow$\" in HAKE with \"$\\models$\" for mathematical rigor. The \"$\\rightarrow$\" in appendix Tab. 2 should be corrected as \"$\\models$\".\nWe will add explanations and correct notations in the final version.\n\n\n### Literature Review of Symbolic Reasoners\n\nCompared with standard first-order logic:\n\n- We follow the principles of the standard first-order logic (FoL), e.g., logical connectives $\\wedge, \\vee$. The standard FoL is typically built upon a simplified world model, where symbols are definitely True/False. However, when adapting FoL in vision reasoning tasks, the symbols (e.g., $m_1,...m_{10}$ in Fig. 6) are not known to be True/False. Instead, the models are required to predict the symbol probabilities (usually via a neural network). Thus, a hybrid \"neuro-symbolic\" method should be adopted.\n\nCompared with other neuro-symbolic reasoners:\n\n- Pros: Compared with reasoners for visual question answering on simplified synthetic datasets [1], our method solves real-world visual activity understanding tasks that involve complex patterns (rules). Compared with other activity reasoning methods [2], we propose a novel symbolic system with broad semantics and rational rules to improve compositional generalization. Also, more explainable and lightweight calculations based on fuzzy-logic is provided.\n- Cons: We mainly focus on the design of a symbolic system, while symbol recognition is another important issue left for more exploration. Please refer to Reviewer zui3 \"System-1/2 Ablation\" and \"Symbol Recognition Subtleties\".\n\nWe will add explanations in the \"Related Work\" section.\n\n[1] Yi, Kexin, et al. \"Neural-symbolic vqa: Disentangling reasoning from vision and language understanding.\" Advances in neural information processing systems 31 (2018).\n\n[2] Li, Yong-Lu, et al. \"Hake: a knowledge engine foundation for human activity understanding.\" IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).\n\n### Writing\n\nThanks for your advice. We will provide more examples and clarify the empirical settings and definitions. \n\nThe empirical setting (main paper L232) is detailed in Appendix L155-159. The final prediction is the weighted sum of two predictions with a calculated re-scaling factor.\n\nIn Def-5, \"activity semantics\" are ground-truth activities happening in an image $I$, and \"ongoing symbols\" are ground-truth symbols happening in the image, which can be used as premises to reason out the activities. For example, in Fig.6 image-I, \"activity semantics\" include \"human buy orange\", and \"ongoing symbols\" include \"talk with seller\", \"seller hand over orange\", etc.\n"}]

Author rebuttal:
We sincerely thank all reviewers for their constructive comments and recognition of our work’s strengths:

- It is interesting (Reviewer K3QA), encouraging, and worth appreciation (Reviewer zui3) to leverage and extend the intuitive idea of the System-1 and System-2 combination. The contributions are beneficial for the community (Reviewer yAvu).
- The methodology and formalism are reasonable, promising (Reviewer yZAX), well described (Reviewer sNaH, SHNq, yZAX), and easy to follow (Reviewer yZAX). The paper conveys its idea with a good presentation using clear visualizations (Reviewer yAvu).
- Experiments obtain interesting results (Reviewer yAvu) and highlight the contribution (Reviewer SHNq).

We hope to address the questions and concerns via our responses for each review. In the final version, explanations will be added to enhance clarity.
For reading convenience, we summarize some of the major questions raised by reviewers and our response here.

- Generality (Reviewer sNaH, zui3)
    - The proposed symbolic system can be generalized to various tasks (e.g., classification, visual commonsense) with a similar formulation. We choose activity understanding as a good and important initial test bed.
- Computation Scalability (Reviewer SHNq)
    - Symbol predictions will increase the computational cost as a trade-off for explainability and generalization. However, with the proposed method design below, the computational cost can be largely reduced and will not be a major limitation.
- Evaluation Baselines (Reviewer sNaH, SHNq)
    - We explain the baselines of recent VLMs and HOI models.
- Detailed Evaluation\&Analysis (Reviewer K3QA, zui3)
    - We evaluate the proposed two ideal properties, analyze the performance bottleneck and investigate the robustness of the generated ontology.

Meta-review: Reviewers noted the strengths of this paper to include the important problem being tackled, the novel and promising method, as well as interesting results.

There were initial concerns or questions about: i) comprehensiveness of evaluations, ii) baseline comparison to VLM/LLM without logical system, iii) scalability and iv) understanding the pipeline/system and its bottlenecks. In response, the authors reported an impressive amount of additional work, including results on CIFAR-100, VCR and Stanford-40, a newer VLM model, analysis of System-1 vs System-2 contributions, scalability analysis, etc. The authors also correctly point out that certain SOTA models were released within 2 months of submission date, and hence by NeurIPS policy, these need not be considered.

Overall, the original submission by itself was already a fairly solid piece of work, and the additional results certainly strengthened and generalized the work further (as opposed to patching up flaws). Particularly to encourage non-mainstream approaches, so as to "help the NeurIPS community move faster out of local minima", this work deserves visibility, and I hope that acceptance to NeurIPS will both benefit the community, as well as encourage the authors to take this line of work further.
Decision: Accept (poster)