{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Notebook which Uses a Single Agent to Answer a Query Regarding Multiple Countries\n", "\n", "A single agent uses its tools to plan out each step when a user ask about ESG policies regarding mutltiple countries. Users can compare policies between 2 or more countries.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "sPoHgiG0Vm1J" }, "outputs": [], "source": [ "import os\n", "from langchain.llms import HuggingFaceHub # for calling HuggingFace Inference API (free for our use case)\n", "from langchain.embeddings import HuggingFaceEmbeddings # to let program know what embeddings the vector store was embedded in earlier\n", "\n", "# to set up the agent and tools which will be used to answer questions later\n", "from langchain.agents import initialize_agent \n", "from langchain.agents import tool # decorator so each function will be recognized as a tool\n", "from langchain.chains.retrieval_qa.base import RetrievalQA # to answer questions from vector store retriever\n", "# from langchain.chains.question_answering import load_qa_chain # to further customize qa chain if needed\n", "from langchain.vectorstores import Chroma # vector store for retriever\n", "import ast # to parse user string input to list for one of the tools (agent tools do not support 2 inputs)\n", "#from langchain.memory import ConversationBufferMemory # not used as of now\n", "import pickle # for loading the bm25 retriever\n", "from langchain.retrievers import EnsembleRetriever # to use chroma and \n", "\n", "# for defining a generic LLMChain as a generic chat tool (if needed)\n", "from langchain.prompts import PromptTemplate \n", "from langchain.chains import LLMChain\n", "\n", "import warnings\n", "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n", "warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n", "\n", "# os.environ['HUGGINGFACEHUB_API_TOKEN'] = 'your_api_key' # for using HuggingFace Inference API\n", "# alternatively set your env variable above" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Configuration\n", "\n", "This configuration could be changed in the main app." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "chunk_size = 1000 # choose one of [500, 600, 700, 800, 900, 1000, 1250, 1500, 1750, 2000, 2250, 2500, 2750, 3000]\n", "chunk_overlap = 100 # choose one of [50, 100, 150, 200]\n", "\n", "chroma_n_similar_documents = 5 # number of chunks returned by chroma vector store retriever\n", "bm25_n_similar_documents = 5 # number of chunks returned by bm25 retriever\n", "\n", "# llm for HuggingFace Inference API\n", "# model = \"mistralai/Mistral-7B-Instruct-v0.2\"\n", "model = \"mistralai/Mixtral-8x7B-Instruct-v0.1\"\n", "\n", "# what retrievers to use, if ensemble is chosen then both semantic and keyword similarity documents will be used and results reranked\n", "retriever_config = 'ensemble' # choose one of ['semantic', 'keyword', 'ensemble']\n", "keyword_retriever_weight = 0.3 # choose between 0 and 1\n", "semantic_retriever_weight = 1 - keyword_retriever_weight\n", "\n", "# llm configuration\n", "temperature = 0.25\n", "max_new_tokens = 500\n", "\n", "# in main app, add configuration for user to scrape new data from DuckDuckGo\n", "# in main app, add configuration for user to upload PDF to override country's existing policies in vectorstore" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Countries of Interest\n", "\n", "- This is the list of countries present in the vectore store, since the vector store is prepared from the previous Scrape_Web_and_Process_Data_for_Countries_ESG_Policies notebook, it should be the same list of countries in that notebook. \n", "- This is for coding the RetrievalQA tool later to check, because even if the country given to it is not in the vector store, it would still filter the vector store with this country and give an empty result, instead of giving an error. We have to manually return the error to let the agent using the tool know.\n", "- The countries were reduced to just 6 as the time taken to get the embeddings to build up the chunks is too long. However, having more countries **will not affect** the quality of the answers in comparing between 2 countries in the RAG application as the RAG only picks out document chunks for the 2 countries of interest.\n", "\n", "- In the main app, user can choose which countries to scrape data for, and this list would be adjusted accordingly." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "id": "I29ySuGwgevv" }, "outputs": [], "source": [ "countries = [\n", " \"Australia\",\n", " \"China\",\n", " \"Japan\",\n", " \"Malaysia\",\n", " \"Singapore\",\n", " \"Germany\",\n", " ]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# LLM from HuggingFace Inference Endpoint API" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "id": "ArawtMKUtTtH" }, "outputs": [], "source": [ "# This is an inference endpoint API from huggingface, the model is not run locally, it is run on huggingface\n", "# It is a free API that is very good for deploying online for quick testing without users having to deploy a local LLM\n", "llm = HuggingFaceHub(repo_id=model,model_kwargs={'temperature':temperature,\"max_new_tokens\":max_new_tokens})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Initialize Chroma Vector Store\n", "\n", "As per the Scrape_Web_and_Process_Data_for_Countries_ESG_Policies notebook. Chromadb vector stores have **already been pre-created** for the countries above for each of the different chunk sizes and overlaps, to save time when experimenting as the embeddings take a long time to generate. When using the existing vector stores, we just need to change the name of the persist directory when selecting the different chunk sizes and overlaps.\n", "\n", "Not in this notebook: Later in the main app if the **user choose to scrape new data, or override with their own PDF, a new chromadb would be created**." ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [], "source": [ "# We use HuggingFaceEmbeddings() as it is open source and free to use.\n", "# Initialize the default hf model for embedding the tokenized texts into vectors with semantic meanings\n", "hf_embeddings = HuggingFaceEmbeddings()\n", "\n", "persist_directory = f\"chromadb/chromadb_esg_countries_chunk_{chunk_size}_overlap_{chunk_overlap}\"\n", "# in main app this chroma_db\n", "chroma_db = Chroma(persist_directory=persist_directory,embedding_function=hf_embeddings)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Initialize BM25 Retriever\n", "\n", "- BM25 is a keyword-based algorithm that performs well on queries containing keywords without capturing the semantic meaning of the query terms, hence there is no need to embed the text with HuggingFaceEmbeddings and it is relatively faster to set up. The retrievers with different chunking sizes and overlaps and countries are saved as pickle files in the earlier notebook.\n", "\n", "- Need to initialize one BM25Retriever for each country so the search results later in the main app can be limited to just a particular country. (Chroma DB gives an option to filter metadata for just a particular country during the retrieval processbut BM25 does not because it makes use of external ranking library.) A separate retriever was saved for each country in the earlier notebook." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [], "source": [ "bm25_retrievers = {}\n", "\n", "for country in countries:\n", " bm25_filename = f\"bm25/bm25_esg_countries_{country}_chunk_{chunk_size}_overlap_{chunk_overlap}.pickle\"\n", " with open(bm25_filename, 'rb') as handle:\n", " bm25_retriever = pickle.load(handle) \n", " bm25_retriever.k = bm25_n_similar_documents\n", " bm25_retrievers[country] = bm25_retriever" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Potential Tools for LLM Agent to Use\n", "\n", "Please see comments above each tool. The first tool is the most important one." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "# The most important tool is the first one, which uses a RetrievalQA chain to answer a question about a specific country's ESG policies, \n", "# e.g. carbon emissions policy of Singapore. \n", "# By calling this tool multiple times, the agent is able to look at the responses from this tool for both countries and compare them. \n", "# This is far better than just retrieving relevant chunks for the user's query and throw everything to a single RetrievalQA chain to process\n", "# Multi input tools are not available, hence we have to prompt the agent to give an input list as a string\n", "# then use ast.literal_eval to convert it back into a list\n", "@tool \n", "def retrieve_answer_for_country(query_and_country: str) -> str: # TODO, change diff chain type diff version answers, change \n", " \"\"\"Gives answer to a query about a single country's public ESG policy.\n", " The input list should be of the following format:\n", " [\"query here which is a complete sentence question\", \"country here which is a single word\"]\n", " The first element of the list is the user query, a complete sentence question surrounded by double quotes.\n", " The second element is the full name of the country involved, surrounded by double quotes, for example \"Singapore\".\n", " The 2 inputs are separated by a comma. Do not write a list comprehension.\n", " The 2 inputs, together, are surrounded by square brackets as it is a list. Remember the square brackets.\n", " Do not put multiple countries into the input at once. Instead use this tool multiple times, one time for each country.\n", " If you have multiple queries to ask about a country, break the query into separate parts and use this tool multiple times, one for each query.\n", " \"\"\"\n", " try:\n", " query_and_country_list = ast.literal_eval(query_and_country)\n", " query = query_and_country_list[0]\n", " country = query_and_country_list[1].capitalize() # in case LLM did not capitalize first letter as filtering for metadata is case sensitive\n", " if not country in countries:\n", " return \"\"\"The country that you input into the tool cannot be found. \n", " If you did not make a mistake and the country that you input is indeed what the user asked,\n", " then there is no record for the country and no answer can be obtained.\"\"\"\n", "\n", " # different retrievers\n", " bm = bm25_retrievers[country] # keyword based\n", " bm.k = bm25_n_similar_documents \n", " chroma = chroma_db.as_retriever(search_kwargs={'filter': {'country':country}, 'k': chroma_n_similar_documents}) # semantic\n", " # ensemble (below) reranks results from both retrievers\n", " ensemble = EnsembleRetriever(retrievers=[bm, chroma], weights=[keyword_retriever_weight, semantic_retriever_weight]) \n", " retrievers = {'ensemble': ensemble, 'semantic': chroma, 'keyword': bm} \n", " \n", " qa = RetrievalQA.from_chain_type(\n", " llm=llm,\n", " chain_type='stuff',\n", " retriever=retrievers[retriever_config], # selected retriever based on user config\n", " return_source_documents=True\n", " )\n", " result = qa(query)\n", " return result['result']\n", " except Exception as e:\n", " return f\"\"\"There is an error using this tool: {e}. Check if you have input anything wrongly and try again. \n", " Remember the 2 inputs, query and country, must both be surrounded by double quotes. \n", " The 2 inputs, together, are surrounded by square brackets as it is a list.\"\"\"\n", "\n", "# if a user tries to casually chat with the agent chatbot, the LLM will be able to use this tool to reply instead\n", "# this is optional, better to let user's know the chatbot is not for casual chatting\n", "@tool\n", "def generic_chat_llm(query: str) -> str:\n", " \"\"\"Use this tool for general queries and casual chat. Forward the user input directly into this tool, do not come up with your own input.\n", " This tool IS NOT FOR MAKING COMPARISONS of anything.\n", " This tool IS NOT FOR FINDING ESG POLICY of any country!\n", " It is only for casual chat! Do not use this tool unnecessarily!\n", " \"\"\"\n", " try:\n", " # Second Generic Tool\n", " prompt = PromptTemplate(\n", " input_variables=[\"query\"],\n", " template=\"{query}\"\n", " )\n", " \n", " llm_chain = LLMChain(llm=llm, prompt=prompt)\n", " return llm_chain.run(query)\n", "\n", " except Exception as e:\n", " return f\"\"\"There is an error using this tool: {e}. Check if you have input anything wrongly and try again. \n", " If you have already tried 2 times, do not try anymore, there is no response for your input.\n", " Move on to the next step of your plan.\"\"\"\n", "\n", "# sometimes the agent will suddenly ask for a 'compare' tool even though it was not given this tool\n", "# hence I have decided to give it this tool that gives a prompt to remind it to look at past information\n", "# and decide whether it is time to darw a conclusion\n", "# tools cannot have no input, hence I let the agent input a 'query' parameter even though it is not used\n", "# having the query as input let the LLM 'recall' what is being asked \n", "# instead of it being lost all the way at the start of the ReAct process\n", "@tool\n", "def compare(query:str) -> str: \n", " \"\"\"Use this tool to give you hints and instructions on how you can compare between policies of countries.\n", " Use this tool as a final step, only after you have used other tools to obtain all the information you need.\n", " When putting the query into this tool, look at the entire query that the user has asked at the start,\n", " do not leave any details in the query out.\n", " \"\"\"\n", " return f\"\"\"Once again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\"\"\"" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [], "source": [ "agent = initialize_agent(\n", " [retrieve_answer_for_country, compare], # tools\n", " #[retrieve_answer_for_country, generic_chat_llm, compare],\n", " llm=llm,\n", " agent=\"zero-shot-react-description\", # this is good\n", " verbose=True,\n", " handle_parsing_errors=True,\n", " return_intermediate_steps=True\n", " # memory=ConversationBufferMemory(\n", " # memory_key=\"chat_history\", return_messages=True\n", " # ),\n", " # max_iterations=10\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Test Functions\n", "\n", "#### These functions are used for testing the vector store retriever and RetrievalQA chain. They are not needed in the main app, but are frequently used when debugging or testing if the agent's answers are making sense." ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [], "source": [ "# for testing, not needed in main app\n", "def retrieve_docs(query, country):\n", " # different retrievers\n", " bm = bm25_retrievers[country] # keyword based\n", " bm.k = bm25_n_similar_documents \n", " chroma = chroma_db.as_retriever(search_kwargs={'filter': {'country':country}, 'k': chroma_n_similar_documents}) # semantic\n", " ensemble = EnsembleRetriever(retrievers=[bm, chroma], weights=[keyword_retriever_weight, semantic_retriever_weight]) # ensemble reranks results from both retrievers\n", " retrievers = {'ensemble': ensemble, 'semantic': chroma, 'keyword': bm} \n", " retriever=retrievers[retriever_config] # selected retriever based on user config\n", " return retriever.get_relevant_documents(query)" ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [], "source": [ "# for testing, not needed in main app\n", "def retrieve_ans(query, country):\n", " # different retrievers\n", " bm = bm25_retrievers[country] # keyword based\n", " bm.k = bm25_n_similar_documents \n", " chroma = chroma_db.as_retriever(search_kwargs={'filter': {'country':country}, 'k': chroma_n_similar_documents}) # semantic\n", " ensemble = EnsembleRetriever(retrievers=[bm, chroma], weights=[keyword_retriever_weight, semantic_retriever_weight]) # ensemble reranks results from both retrievers\n", " retrievers = {'ensemble': ensemble, 'semantic': chroma, 'keyword': bm} \n", " qa = RetrievalQA.from_chain_type(\n", " llm=llm,\n", " chain_type='stuff',\n", " retriever=retrievers[retriever_config], # selected retriever based on user config,\n", " return_source_documents=True\n", " )\n", " result = qa(query)\n", " return result['result']" ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "' I apologize, but the provided context does not contain information about racial harmony policies in Singapore. The context mainly discusses various topics such as the Green Government, the Singapore Green Bond Framework, ESG regulations and policies, and the sustainability governance of PUB, but it does not mention racial harmony policies.'" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# for testing, not needed in main app\n", "retrieve_ans(\"What are the racial harmony policies in Singapore?\", \"Singapore\") # should answer don't know" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[Document(page_content='are indirect emissions that result from the generation of purchased or acquired electricity, heating, cooling and steam consumed by the organisation. Scope 2 emission factors for the calculation of FY2019 and FY2020 electricity consumption in Singapore are taken from the Singapore Energy Statistics 2019 published by the Energy Market Authority in Singapore (0.4085 kilograms CO2/kWh), while calculation of FY2021 electricity consumption in Singapore is taken from the Singapore Energy statistic 2020 published by the Energy Market Authority in Singapore (0.408 kilograms CO2/kWh).• Scope 3 emissions are all other indirect emissions that are a consequence of the activities of the organisation, but occur from sources not owned or controlled by the organisation. For PUB’s report, they include emissions from diesel use and electricity consumption at NEWater and Desalination Design, Build, Own and Operate (DBOO) Facilities, and emissions from the incineration of wastewater sludge.Waste', metadata={'country': 'Singapore', 'page': 29, 'source': 'https://www.pub.gov.sg/-/media/PUB/PDF/PUB_SR_2022_Final.pdf'}),\n", " Document(page_content='in cases where agencies are co-located.4 Formerly known as the ‘Public Sector Taking the Lead in Environmental Sustainability’ (PSTLES) initiative, GreenGov.SG was introduced in 2021 as a key enabler of the Singapore Green Plan 2030. Under GreenGov.SG, the public sector has committed to achieving net zero emissions around 2045, five years ahead of Singapore’s national target of net zero by 2050, reducing energy and water use by 10% and waste disposed of by 30% respectively in 2030. The public sector has put in measures to green our buildings and vehicle fleets, reduce waste, and increase solar energy adoption. These will lead to improvements in performance data in the coming years.Report Findings5 In FY2022, the public sector emitted 3.7 million tonnes of carbon dioxide equivalent. This is about 5.3 per cent lower than the FY2020 baseline. This decrease was mainly due to the decommissioning of the Tuas Incineration Plant in early 2022. The reduction in Scope 1 emissions from this', metadata={'country': 'Singapore', 'language': 'en', 'source': 'https://www.mse.gov.sg/resource-room/category/2023-12-15-press-release-inaugural-greengovsg-report', 'title': 'Ministry of Sustainability and the Environment releases inaugural GreenGov.SG report'}),\n", " Document(page_content='of the Tuas Incineration Plant in early 2022. The reduction in Scope 1 emissions from this facility outweighed the increase in electricity use and the associated Scope 2 emissions across the public sector due to the post-COVID-19 re-opening of the economy.6\\tOverall, public sector emissions are expected to rise and peak around 2025, as more public infrastructure projects, including transportation, healthcare, waste management and water treatment facilities, are completed. As more of the envisaged emission reduction strategies get implemented, long term emissions are expected to reduce and trend towards net zero emissions around 2045.7\\tCommenting on the inaugural report, Minister for Sustainability and the Environment, Ms Grace Fu said, “Singapore is among the earliest governments in the world to publish a report of the public sector’s environmental sustainability performance. This signifies our serious commitment for the Government to be part of our action climate agenda and deliver on', metadata={'country': 'Singapore', 'language': 'en', 'source': 'https://www.mse.gov.sg/resource-room/category/2023-12-15-press-release-inaugural-greengovsg-report', 'title': 'Ministry of Sustainability and the Environment releases inaugural GreenGov.SG report'}),\n", " Document(page_content='where we aim to develop strategies that are flexible and adaptable as well as complementary to other land use needs. Mitigating carbon emissions and adopting sustainable energy are critical components in PUB’s sustainability journey in light of the challenges brought about by climate change. In October 2022, the Public Sector and Singapore have announced targets to achieve net zero emissions around 2045 and by 2050 respectively to secure a greener and more sustainable living environment for future generations. Under the GreenGov.SG sustainability movement launched in 2021 as part of the Singapore Green Plan 2030, the public sector aims to spearhead sustainability efforts at the national level and has committed to peak carbon emissions by 2025 and increase solar deployment to 1.5GWp by 2030. PUB will leverage renewable energy sources, energy-efficient technologies and innovative carbon capture solutions to deliver on these targets. I am confident that, with the support of the general', metadata={'source': 'https://www.pub.gov.sg/-/media/PUB/PDF/PUB_SR_2022_Final.pdf', 'page': 2, 'country': 'Singapore'}),\n", " Document(page_content='and buildings, and optimise its operations (e.g., operating hours, frequencies, patterns) so that the equipment can be operated at optimal condition.REDUCING CARBON EMISSIONS THROUGH ENERGY EFFICIENCYREMOVING CARBON EMISSIONSWith our current efforts, we expect to abate approximately 90% of our carbon emissions around 2045 through replacing fossil fuel-based energy sources with renewable sources and reducing electricity consumption via energy-efficient technologies and water conservation. To remove the remaining 10% of carbon emissions and realise our target of net-zero emissions around 2045, PUB is working with our research partners to explore the application of carbon capture and removal solutions in our water treatment facilities.While carbon removal is an emerging technology focus area, PUB has started two projects with A*STAR’s Institute of Sustainability for Chemicals, Energy and Environment (ISCE2) and University of California, Los Angeles (UCLA) to explore carbon removal in PUB', metadata={'source': 'https://www.pub.gov.sg/-/media/PUB/PDF/PUB_SR_2022_Final.pdf', 'page': 13, 'country': 'Singapore'}),\n", " Document(page_content='water and NEWater, PUB’s carbon footprint is projected to more than double from our current footprint of 279 kt CO2-eq to about 1MT CO2-eq, assuming business-as-usual (BAU). Ensuring resource efficiency is key to reducing excessive carbon emissions which would exacerbate climate change and global temperature rise, and mitigating the impact of higher energy prices wrought by a changing energy landscape. Aligning with the public sector’s net-zero emissions goal as announced in October 2022, PUB is committed to peak our emissions by 2025 and targets to achieve net-zero emissions around 2045, contingent on the progress of technologies and international cooperation to enable mitigation measures. PUB adopts a three-pronged approach to replace, reduce and remove our carbon emissions for a sustainable PUB and actively explores the following solutions to eventually reduce greenhouse gas emissions. Solar energy is clean, renewable and the most promising renewable energy source in Singapore.', metadata={'source': 'https://www.pub.gov.sg/-/media/PUB/PDF/PUB_SR_2022_Final.pdf', 'page': 12, 'country': 'Singapore'}),\n", " Document(page_content='and biodiversity of the surrounding landscape.ACTIVE, BEAUTIFUL, CLEAN WATERS (ABC WATERS) PROGRAMMEFEATURE STORIESActive Beautiful Clean Waters (ABC Waters) at Sungei Ulu PandanGRI [3-3] [302-1] [302-4] [305-1] [305-2] [305-3]• Peak carbon emissions by 2025• Target to achieve net-zero emissions around 2045, contingent on the progress of technologies and international cooperation to enable mitigation measuresCOMMITMENTPERFORMANCEKEY FOCUS AREA 1REDUCING OUR ENERGY AND CARBON FOOTPRINT• Development of floating solar PV systems at Marina, Pandan and Lower Seletar reservoirs• Increasing solar deployment on the rooftops of PUB installations• Procurement of EVs to support PUB operations• Proof-of-concept for CO2Tech and Xylem carbon capture solutions under the Carbon Zero Grand ChallengeLOOKING AHEADThe scope and sources of reported GHG emissions have been revised to align with PUB’s GreenGov.SG Reporting Submissions. N2O emissions are calculated based on IPCC 2006 Guidelines for', metadata={'source': 'https://www.pub.gov.sg/-/media/PUB/PDF/PUB_SR_2022_Final.pdf', 'page': 12, 'country': 'Singapore'}),\n", " Document(page_content='Media & TelecommunicationsVenture CapitalWhite Collar & Investigations PartnersPerspectives Publications CSR Our Commitment to ESGPathwaysGlobalPerspectivesKnowledge HighlightsQ4 2022 Review: Key ESG and public policy developments in Singapore16 January 2023The fourth quarter of 2022 saw Singapore implement a series of initiatives to enhance its efforts in addressing the issue of climate change, and introduce refreshed Industry Transformation Maps (“ITMs”). Among other things, Singapore updated its Nationally Determined Contribution (“NDC”) to reduce emissions and raise its national climate target to achieve net zero by 2050.We are pleased to provide an infographic of these environmental, social and governance (“ESG”) and public policy developments and their potential commercial impact in the following areas.Decarbonisation & ESGSingapore updated its NDC to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (“MtCO2e”) in 2030 after peaking emissions earlier,', metadata={'source': 'https://www.allenandgledhill.com/sg/perspectives/articles/23000/sgkh-q4-2022-review-key-eand-public-policy-developments-in-singapore', 'title': 'Q4 2022 Review: Key ESG and public policy developments in Singapore: Allen & Gledhill', 'description': 'Q4 2022 Review: Key ESG and public policy developments in Singapore', 'language': 'en', 'country': 'Singapore'})]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# for testing, not needed in main app\n", "retrieve_docs(\"What are the carbon emissions in Singapore?\", \"Singapore\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Actual Agent In Action + Test Cases to Compare Against Countries\n", "\n", "Multiple test cases are considered below, for example:\n", "- Comparing between 2 countries\n", "- Between 3 countries (see if tool is called 3 times)\n", "- When country is not present in vector store (ensure no hallucination and agent says don't know)\n", "- When information is not in vector store for both countries (ensure no hallucination and agent says don't know)\n", "- When agent is asked to give opinion, e.g. which is more stringent/effective etc.\n", "- Query asks for multiple queries, for multiple countries\n", " * If query asks for 2 things for 2 countries, tool should be called 2 x 2 = 4 times.\n", " * *Difficult Case: (if query asks for 3 things for 5 countries, tool should ideally be called 3 x 5 = 15 times, the tool did try to do this but failed due to the time limit. Scroll down to the second last test for details. However this is okay if we let the user know not to ask so many things to so many countries at once. Also not advisable to have so many inputs for the LLM to process all at once.*)\n", " * **Difficult Case: (if query asks for 3 things for 3 countries, tool should be called 3 x 3 = 9 times, the agent passed this test although it is still not advisable to ask so many things at once. Scroll down to the last cell to see more.**)\n", "\n", "The list above is not exhaustive." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case: 3 Countries" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to first find out the carbon emission policies for each country. I will use the retrieve_answer_for_country tool for this.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore has committed to achieving net zero emissions by 2050. To reach this goal, the country has updated its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier. Singapore has also launched the National Hydrogen Strategy and established the Sustainable Finance Advisory Panel to build a credible and vibrant sustainable finance ecosystem. In the manufacturing sector, the Economic Development Board has launched refreshed Industry Transformation Maps (ITMs) to grow value-added manufacturing by 50% from 2020 to 2030. The public sector has committed to achieving net zero emissions around 2045, five years ahead of Singapore's national target, by reducing energy and water use and increasing solar energy adoption. In the water sector, PUB is working to reduce carbon emissions through energy efficiency and exploring carbon capture and removal solutions in water treatment facilities.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emission policy for Singapore. I will now find out the carbon emission policies for China and Malaysia.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for china?\", \"china\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m China has pledged to peak carbon emission by 2030 and achieve carbon neutrality by 2060. In order to achieve this target, Chinese government has proposed a \"1 + N\" policy framework, including implementation plans and supporting measures for key areas and sectors to reduce their carbon emissions. The Ministry of Finance has also released opinions to establish a financial support system to achieve China's carbon targets, proposing various fiscal tools to boost green development. These policies aim to guide all government decision-making and medium- to long-term development strategies for carbon emission reduction and green development.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emission policy for China. I will now find out the carbon emission policies for Malaysia.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for malaysia?\", \"malaysia\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Malaysia is aiming to achieve net zero GHG (greenhouse gas) emissions by 2050. The country also plans to formulate the ‘National Energy Policy’ under the 12th Malaysia plan, in addition to conducting feasibility studies on carbon pricing, carbon tax and an emissions trading scheme. Furthermore, the government has announced plans to strengthen climate change governance in the country, including the establishment of the National GHG Centre to improve transparency in climate change data and reporting to boost confidence in low carbon investment. Malaysia is also developing a voluntary carbon market (VCM) and a Domestic Emissions Trading Scheme to attract low-carbon investments, boost the credibility and international competitiveness of Malaysian products, and provide Malaysian entities with long-term financial resources.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now have the carbon emission policies for Singapore, China, and Malaysia. I will now compare these policies.\n", "Action: compare\n", "Action Input: [\"are there any similarities or differences between the policies regarding carbon emission for singapore, china, and malaysia?\"]\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n", "Final Answer: All three countries, Singapore, China, and Malaysia, have committed to achieving net zero emissions in the future. Singapore aims to achieve net zero emissions by 2050, China by 2060, and Malaysia by 2050. Singapore has updated its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier. China has pledged to peak carbon emission by 2030 and achieve carbon neutrality by 2060. Malaysia is aiming to achieve net zero GHG (greenhouse gas) emissions by 2050. All three countries have also established various policies and initiatives to support their goals. Singapore has launched the National Hydrogen Strategy and established the Sustainable Finance Advisory Panel to build a credible and vibrant sustainable finance ecosystem. China has proposed a \"1 + N\" policy framework, including implementation plans and supporting measures for key areas and sectors to reduce their carbon emissions. Malaysia has announced plans to strengthen climate change governance in the country, including the establishment of the National GHG Centre to improve transparency in climate change data and reporting to boost confidence in low carbon investment. Malaysia is also developing a voluntary carbon market (VCM) and a Domestic Emissions Trading Scheme to attract low-carbon investments, boost the credibility and international competitiveness of Malaysian products, and provide Malaysian entities with long-term financial resources.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "All three countries, Singapore, China, and Malaysia, have committed to achieving net zero emissions in the future. Singapore aims to achieve net zero emissions by 2050, China by 2060, and Malaysia by 2050. Singapore has updated its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier. China has pledged to peak carbon emission by 2030 and achieve carbon neutrality by 2060. Malaysia is aiming to achieve net zero GHG (greenhouse gas) emissions by 2050. All three countries have also established various policies and initiatives to support their goals. Singapore has launched the National Hydrogen Strategy and established the Sustainable Finance Advisory Panel to build a credible and vibrant sustainable finance ecosystem. China has proposed a \"1 + N\" policy framework, including implementation plans and supporting measures for key areas and sectors to reduce their carbon emissions. Malaysia has announced plans to strengthen climate change governance in the country, including the establishment of the National GHG Centre to improve transparency in climate change data and reporting to boost confidence in low carbon investment. Malaysia is also developing a voluntary carbon market (VCM) and a Domestic Emissions Trading Scheme to attract low-carbon investments, boost the credibility and international competitiveness of Malaysian products, and provide Malaysian entities with long-term financial resources.\n" ] } ], "source": [ "query = \"are there any similarities or differences between the policies regarding carbon emission for singapore and china and malaysia?\"\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case for No Hallucination: One Country Not Present in Vector Store" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to find out the carbon emissions policies of both singapore and sweden. I can use the retrieve_answer_for_country tool to do this. I will start with singapore.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emissions policy of singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore has updated its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier. The public sector has committed to achieving net zero emissions around 2045, five years ahead of Singapore's national target of net zero by 2050. PUB, Singapore's national water agency, aims to peak its emissions by 2025 and targets to achieve net-zero emissions around 2045. PUB's decarbonisation strategy involves replacing fossil fuel-based energy sources with renewable sources, reducing carbon emissions through energy efficiency and water conservation, and removing carbon through carbon capture, utilisation and removal technologies.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emissions policy of singapore. I will now find out the carbon emissions policy of sweden.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emissions policy of sweden?\", \"sweden\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3mThe country that you input into the tool cannot be found. \n", " If you did not make a mistake and the country that you input is indeed what the user asked,\n", " then there is no record for the country and no answer can be obtained.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m It seems that there is no record for the carbon emissions policy of sweden. I will inform the user about this.\n", "Final Answer: I am unable to find the carbon emissions policy of sweden as there is no record for it. I apologize for the inconvenience.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "I am unable to find the carbon emissions policy of sweden as there is no record for it. I apologize for the inconvenience.\n" ] } ], "source": [ "query = \"are there any differences between the carbon emissions policies of singapore and sweden?\"\n", "\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case for No Hallucination: Information Irrelevant, Not Present for Both Countries" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to first find out what the racial harmony policies are in Germany and Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the racial harmony policy in Germany?\", \"Germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I apologize, but the context provided does not contain any information about a racial harmony policy in Germany.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I need to find out what the racial harmony policies are in Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the racial harmony policy in Japan?\", \"Japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I apologize, but the provided context does not mention any racial harmony policy in Japan. It discusses topics such as human rights due diligence guidelines, social bonds, and sustainable finance policy updates, but there is no information about a racial harmony policy.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I cannot compare racial harmony policies in Germany and Japan because I do not have any information about them.\n", "Final Answer: I'm sorry, but I cannot compare racial harmony policies in Germany and Japan because I do not have any information about them.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "I'm sorry, but I cannot compare racial harmony policies in Germany and Japan because I do not have any information about them.\n" ] } ], "source": [ "query = \"are racial harmony policies in germany better than in japan?\"\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case for No Hallucination: Vague question where exact numbers cannot be easily found" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to find out how much money each country is putting into ESG policies.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"how much money is singapore putting into ESG policies?\", \"Singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I don't have the exact figures for the amount of money Singapore is putting into ESG policies. However, the Singapore Green Plan 2030, which is the country's comprehensive long-term plan to advance sustainable development, outlines various initiatives and targets to be achieved by 2030. The plan includes investments in renewable energy, green buildings, sustainable urban planning, and other ESG-related areas. Additionally, the Monetary Authority of Singapore (MAS) has introduced disclosure and reporting requirements for ESG funds, which suggests a commitment to promoting ESG practices in the financial sector. It would be best to check the official sources or the Singaporean government's website for the most accurate and up-to-date information on the financial commitments to ESG policies.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know how much Singapore is putting into ESG policies. Now I need to find out how much Japan is putting into ESG policies.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"how much money is japan putting into ESG policies?\", \"Japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The Government Pension Investment Fund (GPIF) in Japan manages around JPY200 trillion (approximately USD1.3 trillion) and has committed to the Principles for Responsible Investment (PRI) in 2015, which has significantly impacted other investors and promoted ESG in Japan. However, the text does not provide specific information on the amount of money Japan is putting into ESG policies beyond the GPIF's commitment to the PRI.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know how much Japan is putting into ESG policies. Now I need to compare the amount of money each country is putting into ESG policies.\n", "Action: compare\n", "Action Input: \"is singapore or japan putting more money into esg policies?\"\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: Based on the information provided, it is not possible to definitively say which country is putting more money into ESG policies. Singapore has the Singapore Green Plan 2030, which outlines various initiatives and targets related to ESG, while Japan's Government Pension Investment Fund has committed to the Principles for Responsible Investment. However, the observations do not provide specific financial commitments to ESG policies for either country. It would be best to check the official sources or the governments' websites for the most accurate and up-to-date information on the financial commitments to ESG policies.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "Based on the information provided, it is not possible to definitively say which country is putting more money into ESG policies. Singapore has the Singapore Green Plan 2030, which outlines various initiatives and targets related to ESG, while Japan's Government Pension Investment Fund has committed to the Principles for Responsible Investment. However, the observations do not provide specific financial commitments to ESG policies for either country. It would be best to check the official sources or the governments' websites for the most accurate and up-to-date information on the financial commitments to ESG policies.\n" ] } ], "source": [ "query = \"is singapore or japan putting more money into esg policies?\"\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case: Agent is careful in commenting which measures are more effective\n", "\n", "Instead of giving a baseless comparison, agent comments that more data is needed." ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m To answer this question, I need to first find out how Singapore and Germany aim to control pollution. Then I need to find out which country is more effective in this area. I can use the retrieve_answer_for_country tool to find out how each country aims to control pollution. Then I can use the compare tool to compare the effectiveness of their policies.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"How does Singapore aim to control pollution?\", \"Singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore aims to control pollution through various strategies and initiatives. One key approach is by promoting sustainable energy and reducing carbon emissions. For instance, PUB, the national water agency, is working on mitigating carbon emissions and adopting sustainable energy in its operations. They aim to develop strategies that are flexible and adaptable, as well as complementary to other land use needs. Specifically, they are focusing on renewable energy sources, energy-efficient technologies, and innovative carbon capture solutions to deliver on national targets for net-zero emissions. Additionally, Singapore has also announced targets to achieve net-zero emissions around 2045 and by 2050, respectively, to secure a greener and more sustainable living environment for future generations. Under the GreenGov.SG sustainability movement, the public sector aims to spearhead sustainability efforts at the national level and has committed to peak carbon emissions by 2025 and increase solar deployment to 1.5GWp by 2030.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I know how Singapore aims to control pollution, I need to find out how Germany aims to control pollution.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"How does Germany aim to control pollution?\", \"Germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Germany's Sustainable Development Strategy includes measures to control environmental pollution, protect biodiversity, and maintain ecosystems. The strategy covers various areas, including greenhouse gas reductions, funding for infrastructure, e-vehicles, and international cooperation projects. Germany has also issued Green German Federal securities to enhance transparency and create incentives for sustainable finance. The strategy emphasizes the importance of considering ESG (Environmental, Social, and Governance) factors in corporate decision-making, including the consequences for human rights, climate change, and the environment. However, the document does not provide specific details on how Germany aims to control pollution.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I know how both countries aim to control pollution, I need to compare their effectiveness. I can use the compare tool for this.\n", "Action: compare\n", "Action Input: [\"Compare the effectiveness of Singapore and Germany in controlling pollution, given that Singapore aims to control pollution through various strategies and initiatives, including promoting sustainable energy and reducing carbon emissions, and Germany aims to control pollution through the Sustainable Development Strategy, which includes measures to control environmental pollution, protect biodiversity, and maintain ecosystems, but does not provide specific details on how Germany aims to control pollution.\", \"Singapore, Germany\"]\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: Based on the information provided, Singapore and Germany both have strategies in place to control pollution. Singapore aims to control pollution through various strategies and initiatives, including promoting sustainable energy and reducing carbon emissions. Germany, on the other hand, has a Sustainable Development Strategy that includes measures to control environmental pollution, protect biodiversity, and maintain ecosystems, although it does not provide specific details on how Germany aims to control pollution. Without specific details on Germany's strategy, it is difficult to make a direct comparison between the two countries' effectiveness in controlling pollution. However, Singapore's specific targets for net-zero emissions and solar deployment, as well as its commitment to peak carbon emissions by 2025, suggest a more proactive approach to controlling pollution compared to Germany's more general strategy.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "Based on the information provided, Singapore and Germany both have strategies in place to control pollution. Singapore aims to control pollution through various strategies and initiatives, including promoting sustainable energy and reducing carbon emissions. Germany, on the other hand, has a Sustainable Development Strategy that includes measures to control environmental pollution, protect biodiversity, and maintain ecosystems, although it does not provide specific details on how Germany aims to control pollution. Without specific details on Germany's strategy, it is difficult to make a direct comparison between the two countries' effectiveness in controlling pollution. However, Singapore's specific targets for net-zero emissions and solar deployment, as well as its commitment to peak carbon emissions by 2025, suggest a more proactive approach to controlling pollution compared to Germany's more general strategy.\n" ] } ], "source": [ "query = \"How does Singapore and Germany aim to control pollution, which country is more effective in this area?\"\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Case: Agent is careful in commenting which measures are more stringent" ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to find out the ESG regulations of each country first before comparing them.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"What are the ESG regulations in Singapore?\", \"Singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In Singapore, the Monetary Authority of Singapore (MAS) has introduced a set of guidelines for financial institutions to incorporate ESG factors into their investment decision-making process. Additionally, the MAS has established disclosure and reporting requirements for environmental, social and governance (ESG) funds. These regulations require ESG funds to disclose information on an ongoing basis and provide yearly updates on the progress of the ESG goals that the funds have specifically set. The Singapore Green Plan 2030 is a government initiative that outlines the country's sustainability goals over the next decade, including targets to increase the adoption of renewable energy, reduce carbon emissions, improve waste management, and enhance biodiversity.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I know the ESG regulations in Singapore, I need to find out the ESG regulations in China.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"What are the ESG regulations in China?\", \"China\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In China, there have been several recent developments in ESG regulations and policies. The \"Draft Amendment to Company Law\" was released in December 2021, which includes new ESG provisions for companies. The environmental disclosure rules and other environmental regulations and policies have also been updated. Additionally, there have been efforts to harmonize green bond regulations in the domestic bond markets and adopt the 100% use-of-proceeds approach in line with the global Green Bond Principles. A white paper has also been published to provide better clarity on national regulatory developments in the ESG space.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I know the ESG regulations in China, I need to find out the ESG regulations in Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"What are the ESG regulations in Japan?\", \"Japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In Japan, the Financial Services Agency (FSA) has proposed a hard law amendment to the Financial Instruments Exchange Act (FIEA) that introduces ESG-specific disclosures for public companies. This new section, \"Sustainability Perspective and Measures,\" is designed to address a company's sustainable development consistent with its management policies and strategies. The amendment will apply to the securities registration statement and annual securities report, which are related to the fiscal year ending on and after March 31, 2023. Additionally, there are regulations related to ESG for investors, such as the Stewardship Code and the Corporate Governance Code, which are not legally binding but are widely followed. The FSA also issued ESG guidelines in 2018 for institutional investors to promote ESG investment. In terms of corporate actions, the Japanese government has established a new organization, the Headquarters for Digital Market Competition, and enacted the Act on Improving Transparency and Fairness of Digital Platforms (TFDPA) in 2021 to address certain issues perceived in the digital market. The TFDPA currently covers online mall operators and requires greater transparency and fairness in transactions in online malls and app stores. Some operators which meet certain category and size criteria have been designated as \"Specified Digital Platform Providers\" and are subject to additional regulations.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I have the ESG regulations for all three countries, I can compare them to determine which one has the most stringent regulations.\n", "Action: compare\n", "Action Input: [\"Compare the ESG regulations in Singapore, China, and Japan\", \"\"]\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: In Singapore, the Monetary Authority of Singapore (MAS) has introduced a set of guidelines for financial institutions to incorporate ESG factors into their investment decision-making process and established disclosure and reporting requirements for environmental, social and governance (ESG) funds. The Singapore Green Plan 2030 outlines the country's sustainability goals over the next decade. In China, there have been several recent developments in ESG regulations and policies, including the \"Draft Amendment to Company Law\" and updated environmental disclosure rules. In Japan, the Financial Services Agency (FSA) has proposed a hard law amendment to the Financial Instruments Exchange Act (FIEA) that introduces ESG-specific disclosures for public companies, and there are regulations related to ESG for investors. Based on the information provided, it is challenging to definitively say which country has the most stringent ESG regulations as each country has different focuses and approaches to ESG. However, Japan's proposed amendment to the FIEA and the existence of the Stewardship Code and the Corporate Governance Code suggest a strong emphasis on ESG disclosures and investor regulations. China's recent developments in ESG regulations and policies also indicate a growing focus on ESG. Singapore's MAS guidelines and the Singapore Green Plan 2030 show a commitment to incorporating ESG factors into investment decision-making and promoting sustainability. Ultimately, the stringency of ESG regulations depends on the specific aspects of ESG one is considering and how one evaluates the regulations.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "In Singapore, the Monetary Authority of Singapore (MAS) has introduced a set of guidelines for financial institutions to incorporate ESG factors into their investment decision-making process and established disclosure and reporting requirements for environmental, social and governance (ESG) funds. The Singapore Green Plan 2030 outlines the country's sustainability goals over the next decade. In China, there have been several recent developments in ESG regulations and policies, including the \"Draft Amendment to Company Law\" and updated environmental disclosure rules. In Japan, the Financial Services Agency (FSA) has proposed a hard law amendment to the Financial Instruments Exchange Act (FIEA) that introduces ESG-specific disclosures for public companies, and there are regulations related to ESG for investors. Based on the information provided, it is challenging to definitively say which country has the most stringent ESG regulations as each country has different focuses and approaches to ESG. However, Japan's proposed amendment to the FIEA and the existence of the Stewardship Code and the Corporate Governance Code suggest a strong emphasis on ESG disclosures and investor regulations. China's recent developments in ESG regulations and policies also indicate a growing focus on ESG. Singapore's MAS guidelines and the Singapore Green Plan 2030 show a commitment to incorporating ESG factors into investment decision-making and promoting sustainability. Ultimately, the stringency of ESG regulations depends on the specific aspects of ESG one is considering and how one evaluates the regulations.\n" ] } ], "source": [ "query = \"Are ESG regulations more stringent in Singapore, China or Japan?\"\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test Cases: Agent gave an opinion on which policies are more stringent" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to first find out the ESG policies for financial institutions in Singapore and China. Then, I will compare the two policies to determine which one is more stringent.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what are the esg policies for financial institutions in singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Financial institutions in Singapore are subject to various ESG policies. The Monetary Authority of Singapore (MAS) has introduced guidelines for financial institutions to incorporate ESG factors into their investment decision-making process. ESG funds are required to disclose information on an ongoing basis and provide yearly updates on the progress of their ESG goals. The MAS has also published the Singapore Green Bond Framework, a governance framework for sovereign green bond issuances. Additionally, the MAS has conducted thematic reviews on the implementation of environmental risk management guidelines and released Information Papers on Environmental Risk Management. The MAS has also published a Circular on Disclosure and Reporting Guidelines for Retail ESG Funds and a consultation paper on introducing mandatory disclosure requirements for financial institutions. Furthermore, the MAS has launched an ESG disclosure platform and implemented a climate overlay programme to tilt its equity investments towards exposures that are less carbon-intensive and more sustainable.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I have the information about Singapore's ESG policies, I need to find out the ESG policies for financial institutions in China.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what are the esg policies for financial institutions in china?\", \"china\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In China, ESG policies for financial institutions include the Guidelines for Green Credit, which encourage banks to actively grant green loans, and the Green Finance Guidelines for Banking and Insurance Sectors, which guide banking and insurance institutions to incorporate environmental, social, and governance requirements into their management processes and comprehensive risk management systems. Additionally, policies for guiding bank loans to small and micro enterprises and rural revitalization have been introduced to encourage financial institutions to fulfill their social responsibilities. After 2018, various ESG-related investment guidelines for asset management products have also been issued.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I have the information about China's ESG policies, I need to compare the two policies to determine which one is more stringent.\n", "Action: compare\n", "Action Input: \"what are the esg policies for financial institutions in singapore and china, and which one is more stringent?\"\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Singapore's ESG policies for financial institutions are more stringent compared to China's. Singapore's policies include guidelines for investment decision-making, disclosure and reporting requirements, a governance framework for sovereign green bond issuances, thematic reviews on environmental risk management, and a climate overlay programme. China's policies include guidelines for green credit, guidelines for banking and insurance sectors, and investment guidelines for asset management products.\n", "Final Answer: Singapore's ESG policies for financial institutions are more stringent compared to China's. Singapore's policies include guidelines for investment decision-making, disclosure and reporting requirements, a governance framework for sovereign green bond issuances, thematic reviews on environmental risk management, and a climate overlay programme. China's policies include guidelines for green credit, guidelines for banking and insurance sectors, and investment guidelines for asset management products.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "Singapore's ESG policies for financial institutions are more stringent compared to China's. Singapore's policies include guidelines for investment decision-making, disclosure and reporting requirements, a governance framework for sovereign green bond issuances, thematic reviews on environmental risk management, and a climate overlay programme. China's policies include guidelines for green credit, guidelines for banking and insurance sectors, and investment guidelines for asset management products.\n" ] } ], "source": [ "query = \"what are the esg policies for financial institutions for both singapore and china? which is more stringent?\"\n", "\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "code", "execution_count": 52, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to first retrieve the ESG policies for financial institutions in Singapore and Germany. After that, I will compare the two policies to determine which is more stringent and why.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what are the esg policies for financial institutions in singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Financial institutions in Singapore are subject to various ESG policies. The Monetary Authority of Singapore (MAS) has introduced guidelines for financial institutions to incorporate ESG factors into their investment decision-making process. ESG funds are required to disclose information on an ongoing basis and provide yearly updates on the progress of their ESG goals. The MAS has also published the Singapore Green Bond Framework, a governance framework for sovereign green bond issuances. Additionally, the MAS has conducted thematic reviews on the implementation of environmental risk management guidelines and released Information Papers on Environmental Risk Management. The MAS has also published a Circular on Disclosure and Reporting Guidelines for Retail ESG Funds and a consultation paper on introducing mandatory disclosure requirements for financial institutions. Furthermore, the MAS has launched an ESG disclosure platform and implemented a climate overlay programme to tilt its equity investments towards exposures that are less carbon-intensive and more sustainable.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I have the ESG policies for financial institutions in Singapore, I need to retrieve the ESG policies for financial institutions in Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what are the esg policies for financial institutions in germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m\n", "\n", "Financial institutions in Germany are subject to specific ESG risk management requirements, as outlined by the German Federal Financial Supervisory Authority (BaFin) in its draft on the minimum requirements for the risk management of credit institutions. These guidelines adopt the corresponding guidelines of the European Banking Authority (EBA) and provide specifications for the inclusion of ESG factors and risks in lending and monitoring. Additionally, BaFin expects credit institutions to adapt their existing processes and develop newer measurement, management, and risk mitigation tools to counter ESG risks.\n", "\n", "The EU Corporate Sustainable Reporting Directive (CSRD) and the EU Taxonomy Regulation are at the heart of ESG regulation in the EU, and they significantly expand the non-financial reporting obligations of undertakings. These regulations will apply to all large undertakings and, from 2026, capital market-oriented SMEs in Germany, increasing the number of obligated undertakings from 550 to 15,000.\n", "\n", "The German government is also taking steps to promote sustainable finance, such as developing its own ESG definitions and standards if none are established, allocating federal investments into sustainable investments, shifting civil-service pension funds to sustainable indices, and continuously reducing the carbon footprint of these portfolios. Additionally, Germany aims to issue green bonds in further maturities to establish a green bond yield curve that will become a benchmark in the eurozone’s green capital market.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Now that I have the ESG policies for financial institutions in both Singapore and Germany, I can compare the two policies to determine which is more stringent and why.\n", "Action: compare\n", "Action Input: \"what are the esg policies for financial institutions in singapore and germany? which is more stringent? why?\"\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n", "Final Answer: The ESG policies for financial institutions in Singapore and Germany differ in their specificity and scope. In Singapore, the Monetary Authority of Singapore (MAS) has introduced guidelines for financial institutions to incorporate ESG factors into their investment decision-making process, with mandatory disclosure requirements and a climate overlay programme. In Germany, the German Federal Financial Supervisory Authority (BaFin) has specific ESG risk management requirements for credit institutions, and the EU Corporate Sustainable Reporting Directive (CSRD) and the EU Taxonomy Regulation significantly expand the non-financial reporting obligations of undertakings. Both countries have taken steps to promote sustainable finance, but Germany has a more comprehensive approach with its own ESG definitions and standards, federal investments into sustainable investments, and shifting civil-service pension funds to sustainable indices. Therefore, Germany's ESG policies for financial institutions can be considered more stringent due to their specificity and scope.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "The ESG policies for financial institutions in Singapore and Germany differ in their specificity and scope. In Singapore, the Monetary Authority of Singapore (MAS) has introduced guidelines for financial institutions to incorporate ESG factors into their investment decision-making process, with mandatory disclosure requirements and a climate overlay programme. In Germany, the German Federal Financial Supervisory Authority (BaFin) has specific ESG risk management requirements for credit institutions, and the EU Corporate Sustainable Reporting Directive (CSRD) and the EU Taxonomy Regulation significantly expand the non-financial reporting obligations of undertakings. Both countries have taken steps to promote sustainable finance, but Germany has a more comprehensive approach with its own ESG definitions and standards, federal investments into sustainable investments, and shifting civil-service pension funds to sustainable indices. Therefore, Germany's ESG policies for financial institutions can be considered more stringent due to their specificity and scope.\n" ] } ], "source": [ "query = \"what are the esg policies for financial institutions for both singapore and germany? which is more stringent? why?\"\n", "\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# `Failed Difficult Test Case`: Three Queries, Five Countries\n", "\n", "### - Tool should ideally be called 3 x 5 = 15 times, it tried to do so but was cut off due to the time and token limit. (best not to increase the time limit just to pass this scenario in case the agent get stuck forever in a long loop in other scenarios). \n", "\n", "### - Also sometimes when the query becomes too long, it will not try to split the query up and ask a combined query to each country.\n", "\n", "### - Okay to fail this case if we let the user not to ask so many things about so many countries all at once." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "id": "SxjET_srVm1b" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to find out the policies on carbon emission, water pollution and company esg disclosures for each country. I will use the retrieve_answer_for_country tool for each country and each policy. I will then compare the policies between the countries.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on carbon emission for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore has updated its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier. The country has also raised its national climate target to achieve net zero by 2050. The public sector aims to peak carbon emissions by 2025 and increase solar deployment to 1.5GWp by 2030. PUB, the national water agency, will leverage renewable energy sources, energy-efficient technologies and innovative carbon capture solutions to deliver on these targets.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on carbon emission for Singapore. I will now find out the policy on water pollution for Singapore.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on water pollution for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore's policy on water pollution involves long-term planning, tackling long-term challenges using innovative and technological solutions, and investing ahead of needs. PUB, Singapore's national water agency, works with various government agencies at the planning stage to site developments with high pollution risk outside water catchment areas. Developers and owners of potentially pollutive developments within water catchment areas are required to implement best management practices to avoid pollution. PUB has also put in place a comprehensive water-quality monitoring system which includes the monitoring of reservoir water.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on water pollution for Singapore. I will now find out the policy on company esg disclosures for Singapore.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on company esg disclosures for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The policy on company ESG disclosures in Singapore is that listed companies are required to disclose their climate-related risks, based on TFCD recommendations, from 2023 onwards. Additionally, the Monetary Authority of Singapore (MAS) is consulting on introducing mandatory disclosure requirements for financial institutions when the international standards set by the International Sustainability Standards Board is finalized. The MAS has also issued a set of disclosure and reporting guidelines for retail ESG funds, which will take effect from January 2023. Furthermore, the MAS and SGX have launched an ESG disclosure platform to enhance transparency and comparability of ESG data.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on company esg disclosures for Singapore. I will now find out the policy on carbon emission for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on carbon emission for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Japan's policy on carbon emissions is to achieve net-zero carbon emissions by 2050. The Green Transformation policy, approved in February 2023, outlines the strategy for meeting this goal. It includes using nuclear power, boosting renewables, and introducing a carbon-pricing mechanism. The policy also considers replacing nuclear reactors with next-generation reactors and launching an emissions trading scheme in the 2026-27 financial year. Additionally, a carbon surcharge for fossil fuel importers will be introduced from the financial year 2028-29.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on carbon emission for Japan. I will now find out the policy on water pollution for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on water pollution for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The provided context does not mention any specific policy on water pollution for Japan. It mainly focuses on the efforts being made by the Japanese government to implement ESG initiatives, with a particular focus on environmental considerations and the goal of net-zero carbon emissions by 2050.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I do not know the policy on water pollution for Japan. I will now find out the policy on company esg disclosures for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on company esg disclosures for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Japan has introduced mandatory ESG disclosures for public companies. The enabling regulations of the Financial Instruments Exchange Act of Japan (FIEA) now have a new section, \"Sustainability Perspective and Measures,\" which introduces ESG-specific disclosures for public companies in Japan. This section includes mandatory disclosure items for \"governance\" and \"risk management,\" and optional disclosure items for \"strategy\" and \"index and target\" if they have materiality against the framework populated for \"governance\" and \"risk management.\" The policy is aimed at providing an additional ESG information source for asset managers investing in Japanese listed companies, amid global developments in the ESG disclosure framework.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on company esg disclosures for Japan. I will now find out the policy on carbon emission for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on carbon emission for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Germany is committed to achieving greenhouse gas neutrality by 2045, with steadily decreasing emission levels specified. The Federal Government passed a Climate Protection Act in 2019, amended in 2021, to this end. Germany is also promoting more rapid implementation of the 13th Global Sustainability Goal (\"climate protection measures\"). The government is also taking steps to shift federation's various civil-service pension funds over time to sustainable indices and pledging to continuously reduce the carbon footprint of these portfolios. Additionally, Germany aims to issue green bonds in further maturities to establish a green bond yield curve that will become a benchmark in the eurozone’s green capital market.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on carbon emission for Germany. I will now find out the policy on water pollution for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on water pollution for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The German government's Sustainable Development Strategy includes measures to address water pollution and protect water resources. The strategy aims to reduce water pollution, improve water quality, and promote sustainable water use. The government also supports international cooperation to address water pollution and protect water resources globally. However, the specific policies and measures to address water pollution in Germany may vary and are subject to change. For the most up-to-date information, it is recommended to consult the German government's official website or contact relevant authorities.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on water pollution for Germany. I will now find out the policy on company esg disclosures for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on company esg disclosures for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In Germany, there are several policies and regulations regarding company ESG (Environmental, Social, and Governance) disclosures. For real economy undertakings and financial institutions, the current disclosure obligations under the German Commercial Code arise if they are capital market-oriented and have more than 500 employees. These undertakings must submit a non-financial statement covering environmental, employee and social issues, as well as disclosure regarding human rights and the fight against corruption. They must also report, according to the EU Taxonomy Regulation, what proportion of their turnover and CAPEX/OPEX can be considered environmentally sustainable. Other important ESG regulations include the Sustainable Finance Disclosure Regulation, the Climate Benchmark Regulation, and the forthcoming Regulation on the transparency and integrity of ESG ratings. At a national level, Germany has established extensive ESG-related obligations for companies through the Supply Chain Due Diligence Act. Large companies should prepare for the CSRD (Corporate Sustainability Reporting Directive), which is to be applied in a staggered manner from 1 January 2024. They should assess which of the requirements are material to their business and develop comprehensive plans for collecting and disclosing relevant information. The applicable principle of so-called double materiality must be observed.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on company esg disclosures for Germany. I will now find out the policy on carbon emission for Malaysia.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on carbon emission for malaysia?\", \"malaysia\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Malaysia has committed to reducing greenhouse gas emissions to 45% of GDP by 2030 in line with the Paris Agreement, and aims to achieve net zero GHG emissions by 2050. The country also plans to formulate the 'National Energy Policy' under the 12th Malaysia plan, and has introduced a voluntary carbon market in Budget 2022. The government has also announced plans to strengthen climate change governance, including the establishment of the National GHG Centre to improve transparency in climate change data and reporting. Bursa Malaysia has proposed changes to the Listing Requirements to improve sustainability disclosures for listed companies.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on carbon emission for Malaysia. I will now find out the policy on water pollution for Malaysia.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on water pollution for malaysia?\", \"malaysia\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I don't have specific information on the policy on water pollution for Malaysia. However, I can tell you that Malaysia has been taking steps towards sustainability and addressing environmental issues, including water pollution. The government has set a series of credible commitments towards attaining environmental, social, and governance (ESG) goals, which likely include policies on water pollution. Additionally, Malaysia is working on reducing its carbon intensity against its gross domestic product (GDP) and increasing renewable energy capacity, which can also impact water pollution policies. I recommend checking the official government websites or contacting local authorities for the most accurate and up-to-date information on water pollution policies in Malaysia.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I do not know the policy on water pollution for Malaysia. I will now find out the policy on company esg disclosures for Malaysia.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on company esg disclosures for malaysia?\", \"malaysia\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The policy on company ESG disclosures in Malaysia is that listed companies are required to report against a mandatory set of factors using the enhanced Sustainability Reporting Framework. This framework supports businesses to adopt international best practices for ESG-related disclosures and requires companies to report against common indicators, promoting standardization of reporting in the region and boosting investor confidence. The policy is being implemented in a phased, multi-year approach with the view to bolster the resilience of listed companies and encourage more investment.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on company esg disclosures for Malaysia. I will now find out the policy on carbon emission for China.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on carbon emission for china?\", \"china\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m China has pledged to peak carbon emission by 2030 and achieve carbon neutrality by 2060. To achieve this target, the Chinese government has proposed a \"1 + N\" policy framework, which includes implementation plans and supporting measures for key areas and sectors to reduce their carbon emissions. The Ministry of Ecology and Environment has also issued the Administrative Measures for Legal Disclosure of Enterprise Environmental Information, which includes carbon dioxide emissions in the scope of information disclosure for the first time. Additionally, China has implemented various fiscal and tax policies to support environmental protection, energy conservation, and comprehensive utilization of resources to promote low-carbon development.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on carbon emission for China. I will now find out the policy on water pollution for China.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on water pollution for china?\", \"china\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m \n", "\n", "China has implemented policies to address water pollution, including the Water Pollution Prevention and Control Law, which was enacted in 1984 and has been amended several times since then. The law sets standards for water quality, establishes penalties for violations, and requires industries to implement measures to prevent and control water pollution. In addition, the government has launched several major initiatives to improve water quality, such as the \"Action Plan for Prevention and Control of Water Pollution\" in 2015, which aims to reduce water pollution in key river basins and improve the overall quality of the country's water resources. The plan includes measures such as strengthening monitoring and enforcement, promoting the use of cleaner production technologies, and increasing public awareness of water pollution issues.\n", "\n", "Sources:\n", "\n", "* Water Pollution Prevention and Control Law of the People's Republic of China (2018 Amendment)\n", "* Action Plan for Prevention and Control of Water Pollution (2015)\n", "* Ministry of Ecology and Environment of the People's Republic of China (2021). China's Environmental Situation in 2020. Retrieved from \n", "* China Water Risk (2018). China's Water Ten Plan: The Final Countdown. Retrieved from \u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the policy on water pollution for China. I will now find out the policy on company esg disclosures for China.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the policy on company esg disclosures for china?\", \"china\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The policy on company ESG disclosures in China is established by the China Securities Regulatory Commission (CSRC) and is aimed at promoting transparency and accountability in environmental, social, and governance (ESG) practices among Chinese companies. The policy requires companies to disclose ESG information in their annual reports, and provides guidelines on the content and format of the disclosures. The guidelines cover areas such as environmental management, labor practices, and board diversity, and are intended to align with international ESG disclosure standards. The CSRC also encourages companies to obtain external assurance for their ESG disclosures, and has established a system for monitoring and enforcing compliance with the disclosure requirements.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "Agent stopped due to iteration limit or time limit.\n" ] } ], "source": [ "query = \"are there any differences or simiarities between the policies on carbon emission, \\\n", "water pollution and company esg disclosures for singapore, japan, germany, malaysia and china?\"\n", "\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# `Passed Difficult Test Case`: Three Queries, Three Countries\n", "\n", "### - Tool should be called 3 x 3 = 9 times, agent passed the test and broke down the question into 3 separate parts to ask for each country 3 separate times. It did not forget any input in the middle.\n", "\n", "### - There was largely no hallucination when checked against the source docs in retriever." ] }, { "cell_type": "code", "execution_count": 55, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m I need to first find out the carbon emission, sea pollution and company esg disclosures policies for each country.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore has committed to achieving net zero emissions by 2050. To reach this goal, the country has implemented various policies and strategies, such as updating its Nationally Determined Contribution (NDC) to reduce emissions to around 60 million tonnes of carbon dioxide equivalent (MtCO2e) in 2030 after peaking emissions earlier, and raising the national climate target. The Ministry of Trade & Industry launched Singapore's National Hydrogen Strategy, and the Monetary Authority of Singapore established the Sustainable Finance Advisory Panel to build a credible and vibrant sustainable finance ecosystem. Additionally, the public sector has committed to achieving net zero emissions around 2045, five years ahead of Singapore's national target. Various measures are being taken to green buildings, vehicle fleets, reduce waste, and increase solar energy adoption in the public sector.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emission policy for Singapore. I will now find out the sea pollution policy for Singapore.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the sea pollution policy for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Singapore's sea pollution policy involves whole-of-government land-use planning, where PUB works with various government agencies at the planning stage to site developments with high pollution risk outside water catchment areas. Developers and owners of potentially pollutive developments, such as farms, that are located within water catchment areas, are required to implement best management practices to avoid pollution. PUB has also put in place a comprehensive water-quality monitoring system which includes the monitoring of reservoir water. Additionally, PUB conducts a continuous review of flood risks based on the latest available climate change data from the Intergovernmental Panel on Climate Change (IPCC) and Centre for Climate Research Singapore (CCRS) to formulate and implement coastal protection measures against long-term sea level rise.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the sea pollution policy for Singapore. I will now find out the company esg disclosures policy for Singapore.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the company esg disclosures policy for singapore?\", \"singapore\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The Monetary Authority of Singapore (MAS) has introduced disclosure and reporting requirements for environmental, social and governance (ESG) funds. In its Circular No. CFC 02/2022, released on 28 July 2022, the central bank announced new guidelines which will enter into force on 1 January 2023. Under the rules, ESG funds will be required to disclose information on an ongoing basis, and investors will receive yearly updates on the progress of the ESG goals that the funds have specifically set. ESG must provide information such as the methodology used to determine the ESG score, the criteria used to select ESG investments, and the process for engaging with investee companies on ESG issues. Companies operating in Singapore should ensure that they are in compliance with these new ESG disclosure requirements.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the company esg disclosures policy for Singapore. I will now find out the carbon emission policy for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Japan's carbon emission policy is to achieve net-zero carbon emissions by 2050. The Green Transformation policy, approved in February 2023, outlines Japan's strategy for meeting this goal. The policy includes using nuclear power, boosting renewables, and introducing a carbon-pricing mechanism. The policy also includes the launch of an emissions trading scheme in the 2026-27 financial year, a voluntary phase for companies for the three-year period ending March 2026, and a carbon surcharge for fossil fuel importers from the financial year 2028-29. Additionally, Japan is considering replacing nuclear reactors scheduled to be decommissioned with next-generation reactors. The government also intends to issue GX Economy Transition Bonds to generate JPY150 trillion until around 2032, which will be redeemed by way of generating funds through a Growth-Intended Carbon Pricing Mechanism, which will consist of both a nationwide carbon credit exchange scheme and a carbon surcharge scheme.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emission policy for Japan. I will now find out the sea pollution policy for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the sea pollution policy for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I don't have information on the sea pollution policy for Japan. The context provided is about the green transformation policy, the Philippines securities market regulator's draft guidelines for sustainability-linked bond issuers, insights into ESG in Japan, and updates on sustainable finance policy in Japan.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I don't have information on the sea pollution policy for Japan. I will now find out the company esg disclosures policy for Japan.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the company esg disclosures policy for japan?\", \"japan\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m The Japanese Financial Services Agency has proposed a hard law amendment that will provide an additional environmental, social, and governance (ESG) information source for asset managers investing in Japanese listed companies. Listed companies in Japan with a current fiscal year end of March 31, 2023, or later will need to prepare their annual public disclosure documents under the new regime. The enabling regulations of the Financial Instruments Exchange Act of Japan (FIEA) are proposed to have a new section, “Sustainability Perspective and Measures,” to introduce ESG-specific disclosures for public companies in Japan. This section will include mandatory disclosures for \"governance\" and \"risk management,\" as well as optional disclosures for \"strategy\" and \"index and target.\" The policy aims to improve the depth and breadth of ESG disclosures by Japanese companies and shift from being voluntary to mandatory.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the company esg disclosures policy for Japan. I will now find out the carbon emission policy for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the carbon emission policy for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m Germany has set a goal to achieve greenhouse gas neutrality by 2045, with steadily decreasing emission levels. The country passed a Climate Protection Act in 2019, which was amended in 2021, to achieve this goal. The Sustainable Development Strategy also promotes more rapid implementation of the 13th Global Sustainability Goal (\"climate protection measures\"). Additionally, the German government has been issuing Green German Federal securities since 2020 to boost sustainability in financial markets.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the carbon emission policy for Germany. I will now find out the sea pollution policy for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the sea pollution policy for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m I don't have real-time access to specific policies, but based on the provided information, Germany's Sustainable Development Strategy includes measures to address environmental pollution, including sea pollution. The strategy aims to protect biodiversity and ecosystems, which would include marine environments. However, for detailed sea pollution policies, I would recommend checking the official website or publications of the German Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the sea pollution policy for Germany. I will now find out the company esg disclosures policy for Germany.\n", "Action: retrieve_answer_for_country\n", "Action Input: [\"what is the company esg disclosures policy for germany?\", \"germany\"]\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3m In Germany, companies are required to disclose ESG (Environmental, Social, and Governance) information under various regulations. The German Commercial Code (Handelsgesetzbuch) requires capital market-oriented companies with more than 500 employees to submit a non-financial statement covering environmental, employee and social issues, human rights, and corruption. The EU Taxonomy Regulation requires disclosure of the proportion of turnover and CAPEX/OPEX that can be considered environmentally sustainable. The Supply Chain Due Diligence Act requires companies with more than 3,000 employees to comply with tiered due diligence requirements within their supply chain from January 1, 2023. The CSRD (Corporate Sustainability Reporting Directive) will require large companies to report on sustainability in a more comprehensive and standardized manner from 2024. Companies should also consider voluntary ESG reporting to meet stakeholder expectations and improve their sustainability performance.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m I now know the company esg disclosures policy for Germany. I will now compare the carbon emission, sea pollution, and company esg disclosures policies for Singapore, Japan, and Germany.\n", "Action: compare\n", "Action Input: [\"compare carbon emission, sea pollution, and company esg disclosures policies for singapore, japan, and germany\"]\u001b[0m\n", "Observation: \u001b[33;1m\u001b[1;3mOnce again, check through all your previous observations to answer the user query. \n", " Make sure every part of the query is addressed by the context, or that you have at least tried to do so.\n", " Make sure you have not forgotten to address anything in the query.\n", " If you still need more details, you can use another tool to find out more if you have not tried using the same tool with the necessary input earlier. \n", " If you have enough information, use your reasoning to answer them to the best of your ability. \n", " Give as much elaboration in your answer as possible but they MUST be from the earlier context.\n", " Do not give details that cannot be found in the earlier context.\u001b[0m\n", "Thought:\u001b[32;1m\u001b[1;3m Singapore has committed to achieving net zero emissions by 2050, Japan aims to achieve net-zero carbon emissions by 2050, and Germany has set a goal to achieve greenhouse gas neutrality by 2045. Singapore's sea pollution policy involves whole-of-government land-use planning, Japan's sea pollution policy is not specified, and Germany's Sustainable Development Strategy includes measures to address environmental pollution, including sea pollution. Singapore, Japan, and Germany have policies to improve company ESG disclosures. Singapore's Monetary Authority of Singapore has introduced disclosure and reporting requirements for ESG funds, Japan's Financial Services Agency has proposed a hard law amendment for ESG-specific disclosures, and Germany requires companies to disclose ESG information under various regulations.\n", "Final Answer: Singapore, Japan, and Germany have committed to achieving net zero carbon emissions by 2050, 2050, and 2045, respectively. Singapore has a sea pollution policy involving whole-of-government land-use planning, Japan's sea pollution policy is not specified, and Germany has measures to address environmental pollution, including sea pollution, in its Sustainable Development Strategy. Singapore, Japan, and Germany have policies to improve company ESG disclosures, with Singapore's Monetary Authority of Singapore introducing disclosure and reporting requirements for ESG funds, Japan's Financial Services Agency proposing a hard law amendment for ESG-specific disclosures, and Germany requiring companies to disclose ESG information under various regulations.\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "Singapore, Japan, and Germany have committed to achieving net zero carbon emissions by 2050, 2050, and 2045, respectively. Singapore has a sea pollution policy involving whole-of-government land-use planning, Japan's sea pollution policy is not specified, and Germany has measures to address environmental pollution, including sea pollution, in its Sustainable Development Strategy. Singapore, Japan, and Germany have policies to improve company ESG disclosures, with Singapore's Monetary Authority of Singapore introducing disclosure and reporting requirements for ESG funds, Japan's Financial Services Agency proposing a hard law amendment for ESG-specific disclosures, and Germany requiring companies to disclose ESG information under various regulations.\n" ] } ], "source": [ "query = \"are there differences regarding carbon emission, \\\n", "sea pollution and company esg disclosures for singapore, japan and germany\"\n", "\n", "result = agent(query)\n", "print(result['output'])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.1" } }, "nbformat": 4, "nbformat_minor": 4 }