The Contributions of Rough Sets
to Artificial Intelligence
Duoqian Miao

Tongji University
Shanghai, P.R. China
May 18, 2024

General Conference Chairs•Pawan Lingras (Saint Mary’s University, Canada)•Dominik Ślęzak (QED Software, Poland)•JingTao Yao (University of Regina, Canada)

Program Committee Chairs•Mengjun Hu (Saint Mary’s University, Canada)•Chris Cornelis (Ghent University, Belgium)•Yan Zhang (California State University San Bernardino, USA)

Honorary Chairs•James Peters (University of Manitoba, Canada)•Andrzej Skowron (Systems Research Institute of Polish Academy of Sciences, Poland)•Yiyu Yao (University of Regina, Canada)

Steering Committee Chairs•Witold Pedrycz (University of Alberta, Canada)•Roman Słowiński (Poznań University of Technology, Poland)•Guoyin Wang (Chongqing University of Posts and Telecommunications, China)2

Outline Artificial Intelligence Rough Set Theory
 Contributions of RS to AI
 Conclusions

3

Artificial Intelligence ChatGPT: Chat Generative Pre-trained Transformer, achat robot program developed by OpenAI. (2022)

 Sora: An AI model that can create realistic andimaginative scenes from text instructions. (2024)[1] https://openai.com/index/chatgpt[2] https://openai.com/index/sora4Artificial Intelligence Knowledge Representation: neural networks
 Knowledge Acquisition: machine learning methods
 Knowledge Application: expert systems

5

Artificial Intelligence Artificial Intelligence research has been based on“Knowledge".
 What is Knowledge?
 Unfortunately: there has been no formal definition of
Knowledge in AI textbooks.

6Outline Artificial Intelligence Rough Set Theory
 Contributions of RS to AI
 Conclusions

7

Rough Set Theory Rough sets (RS) was introduced by Prof.Pawlak in 1982.

 It is an efficient method to handle imprecise,inconsistent and incomplete data based on set theory.
 It has a natural advantage in processing qualitative data.

[1] Z. Pawlak, "Rough Sets," in International Journal of Computer and Information Sciences, vol. 11, pp. 341-356, 1982.
[2] R. Slowinski, C. Zopounidis, "Application of the rough set approach to evaluation of bankruptcy risk," in Intelligent
8
Systems in Accounting, Finance and Management, vol. 4, no. 1, pp.27-41, 1995.

Rough Set Theory
 Input data in rough sets
Fact

Condition attributes

Decision attributeSupportWeather Road

Time

Accident

1

misty

icy

day

yes

6

2

foggy

icy

night

yes

8

3

misty

not icy

night

yes

5

4

sunny

icy

day

no

55

5

foggy

not icy

dusk

yes

11

6

misty

not icy

night

no

15

9

Rough Set Theory
 Output results in rough sets
Decision Rules

cer.

1. (Weather, misty) (Road, icy)→ (Accident, yes)

1.00

2. (Weather, foggy) → (Accident, yes)

1.00

3. (Weather, misty) (Road, not icy)→ (Accident, yes)

0.25

4. (Weather, sunny)→ (Accident, no)1.00

5. (Weather, misty) (Road, not icy)→ (Accident, no)

0.75

10

Rough Set Theory
 Contor Georgy: Set Theory (1883)

 Lotfi Zadeh: Fuzzy Set Theory (1965)

 Zdzislaw Pawlak: Rough Set Theory (1982)

11

Rough Set Theory
 Formal representation of knowledge in RS

K=(U,R)
Where 𝑈 = {𝑥1 , 𝑥2 , ⋯ , 𝑥𝑛 } is the universe,
𝑅 = {𝑅1 , 𝑅2 , ⋯ , 𝑅𝑚 } is a family of equivalence relations.

[1] Z. Pawlak, "Rough Sets," in International Journal of Computer and Information Sciences, vol. 11, pp. 341-356, 1982.
12

The First Contribution
 This marks the first significant contribution of RS to AI,
bridging the gap in the formal definition of knowledge
in AI.

13

Rough Set Theory What is a concept ?
Concept intent ; concept extent

 For a concept, how to formally represent it ?

a Concept ≅ a Subset

14

Rough Set Theory What is a imprecise concept ?
◼ Philosophical viewpoint
A concept is imprecise, if its boundary region is nonempty.

◼ Rough concept
How to represent it?

[1] Ramsey, Frank P. "Tractatus logico-philosophicus." pp. 465-478, 1923.
[2] L. Zadeh, "Fuzzy sets," in Information and Control, vol. 8, no. 3, pp. 338-353, 1965.
[3] J. Yao, A. Vasilakos, W. Pedrycz, "Granular computing: perspectives and challenges," in IEEE Transactions on Cybernetics,
vol. 43, pp. 1977-1989, 2013.
15

Rough Set Theory Rough set model
Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝑋 ⊆ 𝑈, 𝐵 ⊆𝐶, 𝐵(𝑥) is the equivalence class of 𝑥 on 𝐵. The lower and upper
approximations of 𝑋 on 𝐵 are defined as follows:
𝐵∗ (𝑋) = ⋃𝑥∈𝑈 {𝐵(𝑥)|𝐵(𝑥) ⊆ 𝑋},

𝐵∗ (𝑋) = ⋃𝑥∈𝑈 {𝐵(𝑥)|𝐵(𝑥)⋂𝑋 ≠ ∅}.
Then, the boundary region of 𝑋 is 𝐵𝑁𝐵 (𝑋) = 𝐵∗ (𝑋) − 𝐵∗ (𝑋),and 𝑿 is rough when the boundary region is nonempty.
[1] Z. Pawlak, "Rough Sets," in International Journal of Computer and Information Sciences, vol. 11, pp. 341-356, 1982.
16

Rough Set Theory Rough set model
Lower
approximation

Boundary region

A concept

Negative region
17

The Second Contribution
 Rough Set Theory provides a method to
calculate the boundary of a concept, realizing
the computability of the philosopher's decidabledefinition.

18

Rough Set Theory
 Attribute reduction
Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝑋 ⊆ 𝑈,
the postive region of 𝐷 is defined as follows:
𝑃𝑂𝑆𝐵 (𝐷) = ⋃𝑋∈𝑈/𝐷 𝐵∗ (𝑋).

For 𝑏 ∈ 𝐵, the attribute 𝑏 is indispensable in 𝐵, if
𝑃𝑂𝑆𝐵−{𝑏} (𝐷) ≠ 𝑃𝑂𝑆𝐵 (𝐷).

[1] Z. Pawlak, "Rough Sets," in International Journal of Computer and Information Sciences, vol. 11, pp. 341-356, 1982.
19

Rough Set Theory
 Attribute reduction
Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝐵 ⊆𝐶, the 𝐵 is a reduct, if(1) 𝑃𝑂𝑆B (𝐷) = 𝑃𝑂𝑆B (𝐷);(2) For 𝑏 ∈ 𝐵, each attribute is indispensable in 𝐵.

Note that:➢ This is from an algebra viewpoint, it is a decidable definition.
➢ The search of minimum reduction is NP-hard problem.

20

Rough Set Theory
 Reduction results --- 1
Fact

Condition attributes Decision attributeSupportWeatherRoad

Accident

1

misty

icy

yes

6

2

foggy

icy

yes

8

3

misty

not icy

yes

5

4

sunny

icy

no

55

5

foggy

not icy

yes

11

6

misty

not icy

no

15

21

Rough Set Theory
 Reduction results --- 2
Fact

Condition attributes Decision attributeSupportWeatherTime

Accident

1

misty

day

yes

6

2

foggy

night

yes

8

3

misty

night

yes

5

4

sunny

day

no

55

5

foggy

dusk

yes

11

6

misty

night

no

15

22

Rough Set Theory
 Reduction results --- decision rules
Decision Rules

cer.

1. (Weather, misty) (Road, icy)→ (Accident, yes)

1.00

2. (Weather, foggy) → (Accident, yes)

1.00

3. (Weather, misty) (Road, not icy)→ (Accident, yes)

0.25

4. (Weather, sunny)→ (Accident, no)1.00

5. (Weather, misty) (Road, not icy)→ (Accident, no)

0.75

23

The third Contribution
 RS as a machine learning model
It is an efficient tool for handling qualitative data, which
provides multi-granularity representation, learning, andreasoning model in artificial intelligence.
wisdomknowledge
information
data[1] P. Lingras, M. Chen, D. Miao “Rough cluster quality index based on decision theory,” in IEEE Transactions on Knowledge and Data Engineering, vol. 21, pp. 10141026, 2009.
[2] D. Slezak, R. Glick, P. Betliski, et al., A new approximate query engine based on intelligent capture and fast transformations of granulated data summaries, 24
in Journalof Intelligent Information Systems, vol. 50, pp. 385-414, 2018.

Outline Artificial Intelligence Rough Set Theory
 Contributions of RS to AI
 Conclusions

25

On Knowledge Representation
 Knowledge representation
Knowledge

Algebraic
representation

Set theory

Discernibility
matrixInformation
representation

Information
entropy

Zentropy
measure26

Algebraic Representation◼ Discernibility matrix: was proposed by Prof. Skowron in
1992.
Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. The discernibility matrix 𝑀 = [𝑚𝑖𝑗 ]𝑛×𝑛 is deﬁned by the following
equation.
{𝑐 ∈ 𝐶} , 𝑐(𝑥𝑖 ) ≠ 𝑐(𝑥𝑗 ) 𝑎𝑛𝑑 𝐷(𝑥𝑖 ) ≠ 𝐷(𝑥𝑗 )𝑚𝑖𝑗 = ቊ∅ ,𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒[1] A. Skowron, R. Cecylia, “The discernibility matrices and functions in information systems,” in Intelligent decision support: handbook of applications and advances of
the rough sets theory. Dordrecht: Springer Netherlands, pp. 331-362, 1992.
[2] J. Wang, D. Miao, Y. Zhou, ”A review on rough set theory and its applications,” in Pattern Recognition and Artiﬁcial Intelligence, vol. 9, no. 4, pp. 337-344, 27
1996.

Algebraic Representation◼ Attribute Reduction Based on the Discernibility matrix◼ Let 𝑀 is the iscernibility matrix of 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓),
the 𝐵 ⊆ 𝐶 is a reduct , if(1) ∀ ∅ ≠ 𝑚𝑖𝑗 ∈ 𝑀, 𝐵 ∩ 𝑚𝑖𝑗 ≠ ∅;(2) ∀𝑏 ∈ 𝐵, 𝐵 − {𝑏} does not satisfy (1).

Note that:
This is merely a decidable definition for attribute
reduction.
[1] A. Skowron, R. Cecylia, “The discernibility matrices and functions in information systems,” in Intelligent decision support: handbook of applications andadvances of the rough sets theory. Dordrecht: Springer Netherlands, pp. 331-362, 1992.
[2] J. Wang, D. Miao, Y. Zhou, ”A review on rough set theory and its applications,” in Pattern Recognition and Artiﬁcial Intelligence, vol. 9, no. 4, pp. 337-344,
1996.
28

Algebraic Representation
Algorithm 1 Attribute reduction based on discernibility matrix.
Input: A decision table 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓).
Output: A reduct subset of 𝐷𝐼𝑆.
Step 1. Compute the core 𝐶0 = 𝐶𝑂𝑅𝐸𝐷 (𝐶) and 𝑅 = 𝐶0 ;Step 2. Obtain 𝑄 = {𝑚𝑖𝑗 : 𝑚𝑖𝑗 ∩ 𝑅 ≠ ∅, 𝑖 ≠ 𝑗, 𝑖, 𝑗 = 1,2 ⋯ , 𝑛},

𝑀 = 𝑀 − 𝑄, 𝐵 = 𝐴 − 𝑅;
Step 3. ∀𝑎𝑘 ∈ 𝐵, compute its frequency in 𝑀 and set 𝑝(𝑎𝑞 ) = 𝑚𝑎𝑥𝑘 {𝑝(𝑎𝑞 )};Step 4. 𝑅 ← 𝑅 ∪ {𝑎𝑞 };Step 5. Repreat above process until 𝑀 = ∅.
Step 6. Return 𝑅.

[1] J. Wang, R. Wang, D. Miao, Meng Guo, et al., ”Data enriching based on rough set theory,” in Chinese J. Computers, vol. 21, no. 5, pp. 393-400, 1998 .
29

Information Representation
 Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝐵, 𝑃 ⊆ 𝐶,

𝑈/𝐵 = {𝑋1 , 𝑋2 , ⋯ , 𝑋𝑠 },𝑈/𝑃 = {𝑌1 , 𝑌2 , ⋯ , 𝑌𝑞 }.

The probability distribution of 𝐵 and 𝑃 is deﬁned as follows.
𝑋1
𝑋2
⋯
𝑋𝑠
[𝑋; 𝐵] =,𝑃(𝑋1 ) 𝑃(𝑋𝑠 ) ⋯ 𝑃(𝑋𝑠 )
𝑌1
𝑌2
⋯
𝑌𝑞
[𝑌; 𝑃] =,𝑃(𝑌1 ) 𝑃(𝑌2 ) ⋯ 𝑃(𝑌𝑞 )
|𝑌𝑗 |
|𝑋𝑖 |
where 𝑃(𝑋𝑖 ) =, 𝑖 = 1,2, ⋯ , 𝑠, 𝑃(𝑌𝑗 ) =, 𝑗 = 1,2, ⋯ , 𝑞.
|𝑈||𝑈|
[1] D. Miao, ”Rough set theory and and its applications in machine learning,” in Doctoral dissertation,Institute of Automation, Chinese Academy of Sciences, Chinese
academy of sciences, 1997.
30

Information Representation
 Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For
𝐵, 𝑃 ⊆ 𝐶. The information entropy of 𝐵, conditional
entropy of 𝐵 related to 𝑃, and mutual information of 𝐵
and 𝑃 are deﬁned as follows.
𝐻(𝐵) = − σ𝑠𝑖=1 𝑃(𝑋𝑖 )𝑙𝑜𝑔𝑃(𝑋𝑖 ),𝑞𝑠σσ
𝐻(𝐵|𝑃) = − 𝑖=1 𝑃(𝑋𝑖 ) 𝑗=1 𝑃(𝑌𝑗 |𝑋𝑖 ) 𝑙𝑜𝑔𝑃(𝑌𝑗 |𝑋𝑖 ) ,𝐼(𝐵; 𝑃) = 𝐻(𝑃) − 𝐻(𝑃|𝐵).
[1] D. Miao, ”Rough set theory and and its applications in machine learning,” in Doctoral dissertation, Institute of Automation, Chinese academy of sciences, 1997.
31

Information Representation
 Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝐵 ⊆𝐶, the lower and upper approximation can be definedfrom information viewpoint.
𝐵∗ (𝑋) = ⋃𝑥∈𝑈 {𝐵(𝑥)|𝐻(𝑋|𝐵(𝑥)) = 1}
𝐵 ∗ (𝑋) = ⋃𝑥∈𝑈 {𝐵(𝑥)|𝐻(𝑋|𝐵(𝑥)) > 0}
Then, the boundary region of 𝑋 is
𝐵𝑁𝐵 (𝑋) = 𝐵 ∗ (𝑋) − 𝐵∗ (𝑋)𝑋 is rough when the boundary region is nonempty.
[1] D. Miao, ”Rough set theory and and its applications in machine learning,” in Doctoral dissertation, Institute of Automation, Chinese academy of sciences, 1997.
[2] K. Yuan, D. Miao, Y. Yao, et al., ”Feature selection using zentropy-based uncertainty measure,” in IEEE Transactions on Fuzzy Systems, vol. 32, no. 4, pp. 22462260, 2024.
32

Information Representation
 The Attribute Reduction Defined by Information
representation
Let 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓) be a decision table. For 𝐵 ⊆𝐶, the 𝐵 is a reduct subset induced by information entropy, if(1) 𝐻(𝐵) = 𝐻(𝐶);(2) For 𝑏 ∈ 𝐵, 𝐻(𝐵 − {𝑏}) ≠ 𝐻(𝐶).
Subsequently, I developed a heuristic algorithm for
attribute reduction based on Information representation.
[1] D. Miao, ”Rough set theory and and its applications in machine learning,” in Doctoral dissertation, Institute of Automation, Chinese academy of sciences, 1997.
33

Information Representation
Algorithm 2 A heuristic algorithm based on mutual information.
Input: A decision table 𝐷𝐼𝑆 = (𝑈, 𝐶 ∪ 𝐷, 𝑉, 𝑓).
Output: A reduct subset of 𝐷𝐼𝑆.
Step 1. Compute the mutual information 𝐼(𝐶; 𝐷) of 𝐶 and 𝐷.
Step 2. Compute the core 𝐶0 = 𝐶𝑂𝑅𝐸𝐷 (𝐶) of 𝐶 related to 𝐷;
Step 3. Let 𝐵 = 𝐶0 , for ∀𝑝 ∈ 𝐶 − 𝐵 repreats the following:① Compute the conditional mutual information 𝐼(𝑝; 𝐷|𝐵);② Select the attribute satisfying 𝑝0 = 𝑎𝑟𝑔𝑚𝑎𝑥𝑝∈𝐶−𝐵 𝐼(𝑝; 𝐷|𝐵);③ 𝐵 ← 𝐵 ∪ {𝑝0 };④ Stop this step when 𝐼(𝐵; 𝐷) = 𝐼(𝐶; 𝐷), turn to ①
Step 4. Return 𝐵.

[1] D. Miao, G. Hu, ”A heuristic algorithm for reduction of knowledge,” in Journal of Computer research & development, vol. 36, no. 6, pp. 681-684, 1999.

34

Information Representation-New◼ Zentropy was proposed by Liu et al. in 2022.

◼ It emphasizes that system entropy is the whole reflectionof different scales, which have been used for physicalproperty prediction.
◼ We first applied this thought to RS and constructed azentropy-based measure for uncertain knowledge
processing.

[1] Z. Liu, Y. Wang, S. Shang, "Zentropy theory for positive and negative thermal expansion," in Journal of Phase Equilibria and Diffusion, vol. 43, pp. 598-605, 2022.
[2] K. Yuan, D. Miao, Y. Yao, et al., ”Feature selection using zentropy-based uncertainty measure,” in IEEE Transactions on Fuzzy Systems, vol. 32, no. 4, pp. 2246-2260,
35
2024.

Information Representation-New
 The definition of Zentropy𝑚σ
𝑍𝐵 (D) = − σ𝑚
𝑝
𝑙𝑜𝑔𝑝
+
𝑘
𝑘=1 𝑘
𝑘=1 𝑝𝑘 𝑍𝑘 ,current scale entropy lower scale entropy➢ There are 𝑚 parts at the current scale;➢ Each part can be decomposed into thefiner parts at a lower scale.

[1] D. Miao, ”Rough set theory and and its applications in machine learning,” in Doctoral dissertation, Institute of Automation, Chinese academy of sciences, 1997.
[2] K. Yuan, D. Miao, Y. Yao, et al., ”Feature selection using zentropy-based uncertainty measure,” in IEEE Transactions on Fuzzy Systems, vol. 32, no. 4, pp. 224636
2260, 2024.

Comparison of Attribute Reduction Algorithms
 Examples of attribute reduction
Table 1 The reduct resutls of different methods

Representation
Algebraic
representation

Methods

Reducts

Positive region{Weather, Road} and {Weather, time}

Discernibility matrix {Weather, Road} and {Weather, time}
Mutual information {Weather, Road} and {Weather, time}

Information
representation

Zentropy measure {Weather, Road} and {Weather, time}37Comparison of Attribute Reduction Algorithms
 Examples of attribute reduction
Making some changes in the Input :Fact

Condition attributes

Decision attributeSupportWeather

Road

Time

Accident

1

misty

icy

day

yes

6

2

foggy

icy

dusk

yes

8

3

misty

not icy

night

yes

5

4

sunny

icy

day

no

55

5

foggy

not icy

dusk

yes

11

6

misty

not icy

night

no

15
38

Comparison of Attribute Reduction Algorithms
 Examples of attribute reduction
Table 2 The reduct resutls when Input changes

Knowledge

Methods

Reduct subset

Algebraic
representation

Positive region{Weather, Road} and {Weather, time}

Discernibility matrix{Weather, Road} and {Weather, time}

Mutual information{Weather, Road} and {Weather, time}

Zentropy measure{Weather, time}

Information
representation

39

Comparison of Attribute Reduction Algorithms

Compared with other methods, the zentropy measuredescribes the information presented at diﬀerent
granular levels, So, It can accurately character
information changes with attribute variation and obtain the
reducts with Stronger discriminability.

40

Contributions of RS to AI
 Formal definition of knowledge Boundary region calculation Method of imprecise
concept
 A machine learning model for qualitative data
(imprecise, inconsistent, and incomplete)41Outline Artificial Intelligence Rough Set Theory
 Contributions of RS to AI
 Conclusions

42

Conclusions
 Rough sets provides multi-granularity representation,
learning, and reasoning methods for AI.
 Three-way decision(Proposed by Dr. Y.Yao) is an
important extension of RS and has been successfully
applied to various scenarios.

 Rough sets as a data-driven Machine Learning Model
showes strong application value and development
potential.

[1] Y. Yao, "Three-way decisions with probabilistic rough sets," in Information sciences, vol. 180, no. 3, pp.341-353, 2010.
[2] D. Miao, K. Yuan, "The contributions of rough sets to artificial intelligence," submitted to the special issue of 40 Years Rough Sets on Applied Soft Computing
43
Journal.

Paper list in My Lab. (http://iip.tongji.edu .cn)
[1] Y. Zhang, Q. Zhang, Z. Gong, Y. Shi, Y. Liu, D. Miao*, "MLIP: Efficient multi-perspective
language-image pretraining with exhaustive data utilization", ICML 2024.
[2] Z. Wu, Q. Zhang, D. Miao*, "HyDiscGAN: A hybrid distributed cGAN for audio-visual
privacy preservation in multimodal sentiment analysis," IJCAI 2024.
[3] A. Lao, Q. Zhang, C. Shi, L. Cao, K. Yi, L. Hu, D. Miao, "Frequency spectrum is more
effective for multimodal representation and fusion: A multimodal spectrum rumor detector,"
AAAI 2024.
[4] Y. Zhang, Y. Liu, D. Miao*, “MG-ViT: A multi-granularity method for compact and efficientvision transformers," NeurIPS 2023.
[5] Y. Li, Y. Liu, H. Zhang, C. Zhao, Z. Wei, D. Miao*, "Occlusion-aware transformer withsecond-order attention for person re-identification," IEEE Transactions on Image Processing,doi: 10.1109/TIP.2024.3393360, 2024.
[6] K. Yuan, D. Miao*, Y. Yao, H. Zhang, ”Feature selection using zentropy-based uncertainty
measure,” IEEE Transactions on Fuzzy Systems, vol. 32, no. 4, pp. 2246-2260, 2024.
[7] D. Chen, D. Miao*, X. Zhao, "Hyneter: Hybrid network transformer for multiple computer
vision tasks," IEEE Transactions on Industrial Informatics, doi: 10.1109/TII.2024.3367043,
2024.
[8] L. Ying, D. Miao*, Z. Zhang, "3WM-AugNet: A feature augmentation network for remotesensing ship detection based on three-way decisions and multigranularity," IEEE Transactions on
Geoscience and Remote Sensing, vol. 63, pp. 1-19, 2023.
44

Thank you !

45

The Contributions of Rough Sets
to Artificial Intelligence
Duoqian Miao

Tongji University
Shanghai, P.R. China
May 18, 2024