Original: Semantics Extraction and Analytics from SEBI Regulations and Case Files: Industry-academia Collaboration
Revised: Semantics Extraction and Analytics from SEBI Regulations and Case Files: Industry-academia Collaboration

Original: Abstract
Revised: Abstract

Original: Extracting insights from text documents and developing predictive models foranalytics is of critical importance in several domains.
Revised: Extracting insights from text documents and developing predictive models for analytics is of critical importance in several domains.

Original: However, it is a challengingtask owing to the diversity in linguistic characteristics of large scale text corpora,exacerbated by a lack of labeled data.
Revised: However, it is a challenging task owing to the diversity in linguistic characteristics of large scale text corpora, exacerbated by a lack of labeled data.

Original: We present here a case-study on extractingsemantics from complex legal and regulatory documents and applying them toperform analytical tasks such as violation detection and penalty estimation.
Revised: We present here a case-study on extracting semantics from complex legal and regulatory documents and applying them to perform analytical tasks such as violation detection and penalty estimation.

Original: Our system was developed in a joint academic-industry collaboration effort and benefitedfrom their complementary research strengths.
Revised: Our system was developed in a joint academic-industry collaboration effort and benefited from their complementary research strengths.

Original: Specifically, the domain expertiseand problem formulation process in the industrial setting were combined with theexploratory research and experimentation rigor of the educational world to developa system that can help legal actors improve their productivity.
Revised: Specifically, the domain expertise and problem formulation process in the industrial setting were combined with the exploratory research and experimentation rigor of the educational world to develop a system that can help legal actors improve their productivity.

Original: We outline ourcollaboration mechanism, detail the techniques used and functionalities developed,and also discuss the key take-aways that can benefit the research community.
Revised: We outline our collaboration mechanism, detail the techniques used and functionalities developed, and also discuss the key take-aways that can benefit the research community.

Original:  Introduction
Revised:  Introduction

Original: Text documents are an effective communication medium for humans.
Revised: Text documents are an effective communication medium for humans.

Original: A human can read a documentand process it to extract its multiple interpretations, and decide on the relevant interpretation.
Revised: A human can read a document and process it to extract its multiple interpretations, and decide on the relevant interpretation.

Original: However,with a large number of interrelated documents it gets difficult to extract multiplicity of interpretations,and to identify the relevant interpretations (1).
Revised: However, with a large number of interrelated documents it gets difficult to extract multiplicity of interpretations, and to identify the relevant interpretations (1).

Original: An AI system can efficiently perform semanticprocessing of the documents to determine and to ascertain these interpretations.
Revised: An AI system can efficiently perform semantic processing of the documents to determine and to ascertain these interpretations.

Original: Our objective is todevelop a framework to facilitate the processing of semantics from legal documents that originatefrom Securities and Exchange Board of India (SEBI), a regulatory body for the financial marketsin India.
Revised: Our objective is to develop a framework to facilitate the processing of semantics from legal documents that originate from Securities and Exchange Board of India (SEBI), a regulatory body for the financial markets in India.

Original: The regulatory system sees the interplay between regulators, lawyers and companies andrequires solving several analytical tasks such as regulation violation detection, penalty estimation,question and answering, and tracking the biography of regulations.
Revised: The regulatory system sees the interplay between regulators, lawyers and companies and requires solving several analytical tasks such as regulation violation detection, penalty estimation, question and answering and tracking the biography of regulations.

Original: Extracting semantics from legal documents is challenging due to the subjectivity, ambiguity, longform content and complex linguistic patterns prevalent in legal domain.
Revised: Extracting semantics from legal documents is challenging due to the subjectivity, ambiguity, long form content and complex linguistic patterns prevalent in legal domain.

Original: In addition, there areoften no readily available labeled data to train a supervised model and the expected outcomesvary depending on the type of user.
Revised: In addition, there are often no readily available labeled data to train a supervised model and the expected outcomes vary depending on the type of user.

Original: This requires defining a precise scope, formulating the problemappropriately, collecting annotations in a smart manner, designing sophisticated models and evaluatingwith appropriate metrics.
Revised: This requires defining a precise scope, formulating the problem appropriately, collecting annotations in a smart manner, designing sophisticated models and evaluating with appropriate metrics.

Original: No one team can possess the wide-ranging capabilities necessary to developsuch a complex system and collaborations become pertinent.
Revised: No one team can possess the wide-ranging capabilities necessary to develop such a complex system and collaborations become pertinent.

Original: In this work, we outline the collaboration effort between an academic research institute and a financialsector company to develop this semantics extraction and analytics system.
Revised: In this work, we outline the collaboration effort between an academic research institute and a financial sector company to develop this semantics extraction and analytics system.

Original: The two partners providedcomplementary strengths and benefited from their mutual expertise.
Revised: The two partners provided complementary strengths and benefited from their mutual expertise.

Original: For example, the industry partner helped in identifying problems that are relevant for practitioners while the academic partner conducteda variety of experiments using state-of-the-art language models (2).
Revised: For example, the industry partner helped in identifying problems that are relevant for practitioners while the academic partner conducted Broadening Research Collaborations (NeurIPS Workshop 2022). a variety of experiments using state-of-the-art language models (2).

Original: This fruitful collaboration helpedus to accelerate the development of a viable solution, which would have otherwise been complicated.
Revised: This fruitful collaboration helped us to accelerate the development of a viable solution, which would have otherwise been complicated.

Original: Furthermore, we detail the Applied Semantics Extraction and Analytics (ASEA) framework thatperforms the semantic processing of SEBI documents.
Revised: Furthermore, we detail the Applied Semantics Extraction and Analytics (ASEA) framework that performs the semantic processing of SEBI documents.

Original: The functionalities described include anextractive question-answering system to answer queries from a knowledge base of SEBI documents,a mechanism to track amendments to regulations, the creation and automatic assignment of semantictags to sentences and phrases, and a regulatory domain specific language model.
Revised: The functionalities described include an extractive question-answering system to answer queries from a knowledge base of SEBI documents, a mechanism to track amendments to regulations, the creation and automatic assignment of semantic tags to sentences and phrases, and a regulatory domain specific language model.

Original: In summary, this paper highlights the value in a research collaboration that spans outside traditionalacademic circles and provides insights on what works and avenues for improvement.
Revised: In summary, this paper highlights the value in a research collaboration that spans outside traditional academic circles and provides insights on what works and avenues for improvement.

Original: Section 2outlines the collaboration structure, section 3 describes the ASEA framework and section 4 discussesthe take-aways.
Revised: Sectionoutlines the collaboration structure, section 3 describes the ASEA framework and section 4 discusses the take-aways.

Original:  Academia-Industry Collaboration Structure
Revised:  Academia-Industry Collaboration Structure

Original: Large firms deal with a lot of text material and AI systems that can simplify and automate theprocessing of text data is critical to the functioning of businesses.
Revised: Large firms deal with a lot of text material and AI systems that can simplify and automate the processing of text data is critical to the functioning of businesses.

Original: While NLP has made hugeadvances in recent years (3), there are still unique problems that cannot be solved with generalpurpose models and off-the-shelf solutions.
Revised: While NLP has made huge advances in recent years (3), there are still unique problems that cannot be solved with generalpurpose models and off-the-shelf solutions.

Original: Hence formal targeted research is often necessary toaddress the challenges posed by niche problems.
Revised: Hence formal targeted research is often necessary to address the challenges posed by niche problems.

Original: An industrial practitioner must often prioritize forimmediate needs and the resources required for conducting exploratory research is scarce.
Revised: An industrial practitioner must often prioritize for immediate needs and the resources required for conducting exploratory research is scarce.

Original: Hencecollaborating with the universities is a viable alternative.
Revised: Hence collaborating with the universities is a viable alternative.

Original: In this particular effort, industry researchers partnered with the academic faculties to refine their draftproposal and jointly define a problem scope that is both interesting from a research perspective andvaluable from a utilitarian standpoint.
Revised: In this particular effort, industry researchers partnered with the academic faculties to refine their draft proposal and jointly define a problem scope that is both interesting from a research perspective and valuable from a utilitarian standpoint.

Original: This benefited the academic partner by grounding the students’research towards potentially feasible solutions.
Revised: This benefited the academic partner by grounding the students’ research towards potentially feasible solutions.

Original: The collaboration avoided the distraction of problemscoping without actually solving the domain-specific task.
Revised: The collaboration avoided the distraction of problem scoping without actually solving the domain-specific task.

Original: For example, extracting even the basicsemantics from both SEBI regulations and legal case documents was an onerous exercise.
Revised: For example, extracting even the basic semantics from both SEBI regulations and legal case documents was an onerous exercise.

Original: It requiredlegal annotators and domain experts to identify the domain-specific semantics, and developingsophisticated models with limited data was challenging.
Revised: It required legal annotators and domain experts to identify the domain-specific semantics, and developing sophisticated models with limited data was challenging.

Original: In the absence of this partnership, we wouldhave over-complicated the problem with generic solutions disregarding the specific requirements.
Revised: In the absence of this partnership, we would have over-complicated the problem with generic solutions disregarding the specific requirements.

Original: Effective prioritization of the problems is another important area.
Revised: Effective prioritization of the problems is another important area.

Original: Tasks such as semantic parsingof text material or search and navigation for Q&A can be used as a building-block for downstreamapplications and hence were addressed at the beginning.
Revised: Tasks such as semantic parsing of text material or search and navigation for Q&A can be used as a building-block for downstream applications and hence were addressed at the beginning.

Original: A planned order of execution also helped usfocus on problems that have a good impact on the legal community such as penalty estimation andregulation biography, and which are novel and unique in terms of the techniques required to addressthem.
Revised: A planned order of execution also helped us focus on problems that have a good impact on the legal community such as penalty estimation and regulation biography, and which are novel and unique in terms of the techniques required to address them.

Original: Thus, problem formulation, scoping and prioritization are key areas where collaboration iscritical to achieve successful outcomes.
Revised: Thus, problem formulation, scoping and prioritization are key areas where collaboration is critical to achieve successful outcomes.

Original: Evaluation metrics and their applicability is an aspect where the industry perspective can differ fromthe academia.
Revised: Evaluation metrics and their applicability is an aspect where the industry perspective can differ from the academia.

Original: For instance, academic researchers are often interested in obtaining the best availableperformance metric (say F1 score), while the stability and interpretability of the models are criticalfor deployed solutions.
Revised: For instance, academic researchers are often interested in obtaining the best available performance metric (say F1 score), while the stability and interpretability of the models are critical for deployed solutions.

Original: Hence there is an implicit trade-off between the complexity of the model andthe quality of the results.
Revised: Hence there is an implicit trade-off between the complexity of the model and the quality of the results.

Original: The periodic review of the models by the industry partner helped guide themodel developer to focus on research elements beyond traditional metrics and consider the robustnessand explainability dimensions.
Revised: The periodic review of the models by the industry partner helped guide the model developer to focus on research elements beyond traditional metrics and consider the robustness and explainability dimensions.

Original: Constant interaction with industry partners helps the team comprehend the solutions’ practicality,rigor, and utility.
Revised: Constant interaction with industry partners helps the team comprehend the solutions’ practicality, rigor, and utility.

Original: More importantly, the industry comprehended the state of the art’s limitations inaddressing specific problems.
Revised: More importantly, the industry comprehended the state of the art’s limitations in addressing specific problems.

Original: The collaboration was made smooth by a mutual respect for eachpartner’s priority and ideas.
Revised: The collaboration was made smooth by a mutual respect for each partner’s priority and ideas.

Original: The monthly catch-up meetings helped share thoughts, measure progress,identify issues and perform course corrections appropriately.
Revised: The monthly catch-up meetings helped share thoughts, measure progress, identify issues and perform course corrections appropriately.

Original: To over-use the cliché, this collaborationstructure ended up being a win-win for both the parties.
Revised: To over-use the cliché, this collaboration structure ended up being a win-win for both the parties.

Original:  ASEA Framework
Revised:  ASEA Framework

Original: [Figure] Figure 1 shows the interplay between regulators and companies.
Revised: Figure 1 shows the interplay between regulators and companies.

Original: While the essence of the interplaydepicted is applicable across countries, we use specifics applicable to India to illustrate it.
Revised: While the essence of the interplay depicted is applicable across countries, we use specifics applicable to India to illustrate it.

Original: Securitiesand Exchange Board of India, commonly referred to as SEBI, is the regulatory body for securities andcommodity market in India.
Revised: Securities and Exchange Board of India, commonly referred to as SEBI, is the regulatory body for securities and commodity market in India.

Original: The Indian companies are obligated to adhere to the regulations draftedby SEBI.
Revised: The Indian companies are obligated to adhere to the regulations drafted by SEBI.

Original: The regulations need to be interpreted by the information technology departments and/orfinance departments of the companies.
Revised: The regulations need to be interpreted by the information technology departments and/or finance departments of the companies.

Original: Moreover, the companies’ lawyers interact with their SEBIcounterparts about case arguments and outcomes.
Revised: Moreover, the companies’ lawyers interact with their SEBI counterparts about case arguments and outcomes.

Original: A well-formulated AI system can aid and improvethese interpretations and interactions by processing and deriving insights from SEBI regulations,associated case files and other pertinent documents.
Revised: A well-formulated AI system can aid and improve these interpretations and interactions by processing and deriving insights from SEBI regulations, associated case files and other pertinent documents.

Original: Towards this goal, we discuss our frameworktitled Applied Semantics Extraction and Analytics (ASEA).
Revised: Towards this goal, we discuss our framework titled Applied Semantics Extraction and Analytics (ASEA).

Original: 3.1 Semantic Segmentation of Case Files through Sentence Classification
Revised: 3.1 Semantic Segmentation of Case Files through Sentence Classification

Original: Over 1100 Adjudication Orders were scraped from the SEBI website.
Revised: Over 1100 Adjudication Orders were scraped from the SEBI website.

Original: Out of the scraped documents,27 adjudication orders have been annotated with the help of a legal expert giving rise to 2052sentence-label pairs that have been used to train the models.
Revised: Out of the scraped documents, 27 adjudication orders have been annotated with the help of a legal expert giving rise tosentence-label pairs that have been used to train the models.

Original: Keeping the tasks of penalty estimationand violation detection in mind, the annotated adjudication orders only pertain to Prohibitions of
Revised: Keeping the tasks of penalty estimation and violation detection in mind, the annotated adjudication orders only pertain to Prohibitions of

Original: Insider Trading (PIT) regulations.
Revised: Insider Trading (PIT) regulations.

Original: This dataset is the first of its kind for Indian legal adjudicationorders regarding insider trading and can be used by other researchers interested in exploring this lineof work.
Revised: This dataset is the first of its kind for Indian legal adjudication orders regarding insider trading and can be used by other researchers interested in exploring this line of work.

Original: We experimented with a number of machine learning methods to train the sentence classifier.
Revised: We experimented with a number of machine learning methods to train the sentence classifier.

Original: We primarily made use of transformer architectures like uncased L-12 H-768 A-12 BERT, XLNet(4)& Legal BERT, each with a multi layer perceptron that is used on top of the trainable transformerlayers for predicting the label.
Revised: We primarily made use of transformer architectures like uncased L-12 H-768 A-12 BERT, XLNet(4) & Legal BERT, each with a multi layer perceptron that is used on top of the trainable transformer layers for predicting the label.

Original: 3.2 Regulation Violation Detection
Revised: 3.2 Regulation Violation Detection

Original: We pose the problem of regulation violation detection as a multi label classification task.
Revised: We pose the problem of regulation violation detection as a multi label classification task.

Original: Given thefacts of the case and r different regulations, the classification model predicts a one-hot vector oflength R , where dimension r being 1 implies that regulation r is violated.
Revised: Given the facts of the case and r different regulations, the classification model predicts a one-hot vector of length R , where dimension r being 1 implies that regulation r is violated.

Original: As we are interested indetecting regulation violation given a set of facts of the case, we develop a semantic segmentationengine to separate out the different sections of the case-file.
Revised: As we are interested in detecting regulation violation given a set of facts of the case, we develop a semantic segmentation engine to separate out the different sections of the case-file.

Original: We experiment with several different machine learning classification set ups.
Revised: We experiment with several different machine learning classification set ups.

Original: The recent yearshave shown how effective transformer based architectures are for various NLP tasks, and thus wealso develop a transformer based multi label classifier.
Revised: The recent years have shown how effective transformer based architectures are for various NLP tasks, and thus we also develop a transformer based multi label classifier.

Original: However, these models perform poorly ondomain specific tasks such as those in the scientific, medical or legal domain.
Revised: However, these models perform poorly on domain specific tasks such as those in the scientific, medical or legal domain.

Original: A possible solutionin these closed domains is to fine-tune the model on domain specific data, and approaches such as
Revised: A possible solution in these closed domains is to fine-tune the model on domain specific data, and approaches such as

Original: Sci-BERT (5), BioBERT(6) have shown that fine-tuning can greatly improve performance in theseclosed domains.
Revised: Sci-BERT (5), BioBERT(6) have shown that fine-tuning can greatly improve performance in these closed domains.

Original: Thus, we fine-tune BERT for the SEBI domain to create “SEBI-BERT".
Revised: Thus, we fine-tune BERT for the SEBI domain to create “SEBI-BERT".

Original: We finetune on a corpus consisting of SEBI regulations, SEBI case-files as well as a collection of financial and SEBI related news articles.
Revised: We fine tune on a corpus consisting of SEBI regulations, SEBI case-files as well as a collection of financial and SEBI related news articles.

Original: The facts of the case can be large, and thus will not fit in the BERTcontext span.
Revised: The facts of the case can be large, and thus will not fit in the BERT context span.

Original: To solve this, we use the sliding window technique.
Revised: To solve this, we use the sliding window technique.

Original: 3.3 Regulation Biography
Revised: 3.3 Regulation Biography

Original: SEBI provides us information about all regulatory documents including their amended versions andadditional supporting documents related to the domain of investment banking in India.
Revised: SEBI provides us information about all regulatory documents including their amended versions and additional supporting documents related to the domain of investment banking in India.

Original: This mine ofinformation can be best understood when analysed, organised and tagged using various NLP methods.
Revised: This mine of information can be best understood when analysed, organised and tagged using various NLP methods.

Original: Our work offers a comprehensive view of a regulation as well as its metamorphosis over time inthe form of (1)visualisation that identifies what changes have been made between two successiveversions of a regulation document (2)provides additional information extracted from SEBI documentslike Annual reports and Concept papers that help understand the rationale behind the amendmentsmade (3)provides tags in order to categorise the kind of amendments, etc using rule based methods(4)identifies references to the regulations in news Articles and SEBI related case files.
Revised: Our work offers a comprehensive view of a regulation as well as its metamorphosis over time in the form of (1)visualisation that identifies what changes have been made between two successive versions of a regulation document (2)provides additional information extracted from SEBI documents like Annual reports and Concept papers that help understand the rationale behind the amendments made (3)provides tags in order to categorise the kind of amendments, etc using rule based methods (4)identifies references to the regulations in news Articles and SEBI related case files.

Original: It also identifiesreferences of case files in news articles.
Revised: It also identifies references of case files in news articles.

Original: 22091 news articles relevant to SEBI, 4974 news articlesrelevant to SEBI cases and 7406 adjudication orders were used to build this module.
Revised: 22091 news articles relevant to SEBI, 4974 news articles relevant to SEBI cases and 7406 adjudication orders were used to build this module.

Original:  Challenges, Benefits and Learnings
Revised:  Challenges, Benefits and Learnings

Original: We discuss the challenges faced, the benefits of the partnership and our key takeaways below.
Revised: We discuss the challenges faced, the benefits of the partnership and our key takeaways below.

Original: • Lack of Labeled Data : The unavailability of labeled data for evaluation, let alone training was a recurring issue.
Revised: • Lack of Labeled Data : The unavailability of labeled data for evaluation, let alone training was a recurring issue.

Original: The complexity inherent in legal language and the diversity in tasks required thoughtful consultation and careful annotation treatment, available only through wider expertise.
Revised: The complexity inherent in legal language and the diversity in tasks required thoughtful consultation and careful annotation treatment, available only through wider expertise.

Original: • Avalanche Effect : A small change in the input often resulted in massive deviations of the output results.
Revised: • Avalanche Effect : A small change in the input often resulted in massive deviations of the output results.

Original: For instance, two case-files may have large overlap between the facts of the case, but a single difference in the fact can result in one case-file violating a regulation whilethe other not violating.
Revised: For instance, two case-files may have large overlap between the facts of the case, but a single difference in the fact can result in one case-file violating a regulation while the other not violating.

Original: Hence models had to be well calibrated and thoroughly reviewed.
Revised: Hence models had to be well calibrated and thoroughly reviewed.

Original: • Domain Specificity : The legal domain uses precise terminologies and even a small mis- understanding can have enormous consequences.
Revised: • Domain Specificity : The legal domain uses precise terminologies and even a small misunderstanding can have enormous consequences.

Original: Standard NLP techniques often do not perform as well, and domain specific solutions had to be built even for primitive tasks such as entity recognition.
Revised: Standard NLP techniques often do not perform as well, and domain specific solutions had to be built even for primitive tasks such as entity recognition.

Original: These require sustained efforts from a sizable group of researchers.
Revised: These require sustained efforts from a sizable group of researchers.

Original: • Overlapping Tasks :
Revised: • Overlapping Tasks :

Original: Large systems such as ASEA often have overlapping sub-problemsand it is important to design modules that are re-usable.
Revised: Large systems such as ASEA often have overlapping sub-problems and it is important to design modules that are re-usable.

Original: This can amortize the efforts spent on a particular module and improve the overall robustness.
Revised: This can amortize the efforts spent on a particular module and improve the overall robustness.

Original: However, this may result in arriving at a trade-off between performance and complexity.
Revised: However, this may result in arriving at a trade-off between performance and complexity.

Original: • Competing Priorities : With new advances in AI appearing frequently, researchers can get distracted by the urge to consider techniques that are narrow and complicated.
Revised: • Competing Priorities : With new advances in AI appearing frequently, researchers can get distracted by the urge to consider techniques that are narrow and complicated.

Original: On the other hand, industry practitioners may want solutions that are widely applicable.
Revised: On the other hand, industry practitioners may want solutions that are widely applicable.

Original: Collaborations must carefully manage the divergence in priorities and interests.
Revised: Collaborations must carefully manage the divergence in priorities and interests.

Original: • Targeted Solution : Our approach of introducing solutions to specific problems such as penalty estimation, violation detection and regulation biography tracking, all in a framework based on extracting semantics for analytics, can serve as a template for breaking down problems in related areas and indulging in research that has broad interest.
Revised: • Targeted Solution : Our approach of introducing solutions to specific problems such as penalty estimation, violation detection and regulation biography tracking, all in a framework based on extracting semantics for analytics, can serve as a template for breaking down problems in related areas and indulging in research that has broad interest.

Original: • Flexible Planning : When developing complex systems, despite thoughtful approaches, unexpected outcomes are a reality.
Revised: • Flexible Planning : When developing complex systems, despite thoughtful approaches, unexpected outcomes are a reality.

Original: For instance, the number of required annotations may prove to be insufficient or the model doesn’t sufficiently generalize.
Revised: For instance, the number of required annotations may prove to be insufficient or the model doesn’t sufficiently generalize.

Original: Hence any collaboration effort must account for such unknown unknowns , and be flexible and ready to improvise.
Revised: Hence any collaboration effort must account for such unknown unknowns , and be flexible and ready to improvise.

Original: • Complementary Strengths :
Revised: • Complementary Strengths :

Original: The cross-fertilization of problem comprehension in industry with the ability to produce state-of-the-art research in academia can be mutually productive and beneficial.
Revised: The cross-fertilization of problem comprehension in industry with the ability to produce state-of-the-art research in academia can be mutually productive and beneficial.

Original: Hence collaborations benefit from partners with symbiotic pairings.
Revised: Hence collaborations benefit from partners with symbiotic pairings.

Revised: Acknowledgments

Revised: This work has been supported by J.P. Morgan AI Faculty Research Award.

Revised: Any opinions, findings, and conclusions in this paper are those of the authors only and do not necessarily reflect the views of the sponsors.