I seek your assistance in a matter of prompt engineering. I am building an LLM system capable of acting like a super forecaster. The system prompt which is constant and unchanging outlines best practices while the main system uses five separate stepwise prompts in a chain of thought methodology to get best results. However, the system is currently not performing as well as I'd hoped. Here are a few system outputs of the final step, for questions where the model predicted on the final outcome in a greatly incorrect manner (all these questions have already been resolved and show the model was wrong and sometimes highly overconfident).

Analyzing my step 5 analysis and shifts in prediction after discussion with my colleague:

Factor 1: New clade Ib outbreak in Africa Reflection: This factor is highly significant and recent, with information coming from credible sources like WHO and multiple news outlets in August 2024. The spread of a new, potentially more virulent strain of mpox to countries outside Africa (Sweden, Thailand) directly increases the likelihood of the CDC reassessing its risk level. However, the geographical distance between the US and affected African countries mitigates some of the immediate risk.

Factor 2: WHO declaration of Public Health Emergency of International Concern (PHEIC) Reflection: This is a very recent and significant development from August 14, 2024, coming from the highest global health authority. A PHEIC declaration typically prompts increased vigilance and action from national health authorities, which could influence the CDC to reassess its risk levels. However, WHO officials have stated that "mpox is not the new COVID," which might temper the urgency of response.

Factor 3: Current low case numbers in the US Reflection: The most recent data on US mpox cases is from March 15, 2024, which is somewhat outdated given the rapidly evolving global situation. While it provides context for the current US situation, it may not reflect the most recent developments or potential imported cases.

Factor 4: Vaccine effectiveness against new strain Reflection: This factor introduces uncertainty regarding protection against the new clade Ib strain. While the information is limited and from August 2024, it's a relevant consideration that could influence the CDC's risk assessment if evidence emerges of reduced vaccine efficacy.

Factor 5: CDC's current risk assessment for clade I mpox Reflection: The CDC's most recent assessment (July 8, 2024) specifically addressing clade I mpox is highly relevant and relatively recent. It provides a strong basis for maintaining the current risk level, but it predates some significant global developments.

Examining new information from web search results:

Factor 6: CDC's Health Alert Network (HAN) Health Update Reflection: This is a very recent (August 2024) and highly credible source directly from the CDC. It provides updated information about the clade I mpox outbreak and emphasizes the very low risk of importation to the US due to limited travel from affected regions. This factor strongly supports maintaining a low probability of the CDC changing its risk assessment.

Factor 7: US government's preparedness and response Reflection: The CDC's media release from August 22, 2024, outlines extensive preparedness measures, including surveillance systems, diagnostic testing capacity, and vaccine distribution. This recent information from a credible source suggests that the US is well-prepared to detect and manage potential clade I cases, which could influence the CDC to maintain its current risk assessment.

Factor 8: Lack of direct commercial flights and limited travelers Reflection: The CDC's HAN update mentions the limited number of travelers and lack of direct commercial flights from DRC or neighboring countries to the US. This recent information from a credible source significantly reduces the likelihood of clade I mpox importation to the US, supporting a low probability of risk reassessment.

Reflection on potential weaknesses in my analysis:

Recency bias: I may be giving too much weight to the most recent information, particularly the WHO's PHEIC declaration and the spread to new countries. While these are significant developments, they may not necessarily lead to a change in the CDC's risk assessment for the US general population.

Overemphasis on global developments: While the global situation is concerning, I may be underestimating the CDC's focus on US-specific factors when assessing risk to the US general population. The CDC's recent communications emphasize the very low risk to most Americans.

Underestimating US preparedness: I might be undervaluing the extensive preparedness measures outlined by the US government, which could make the CDC more confident in maintaining its current risk assessment.

Potential anchoring: My initial estimate of 12%, later adjusted to 10%, might be anchoring my thinking. The new information, particularly from the CDC, suggests an even lower probability might be warranted.

These weaknesses, particularly the potential underestimation of US preparedness and overemphasis on global developments relative to US-specific factors, warrant a further downward adjustment of my prediction.

Justification: The CDC's assessment of the risk posed by mpox to the US general public is unlikely to exceed "Very Low" before October 1, 2024, for several reasons. Firstly, the CDC's most recent communications, including the Health Alert Network update from August 2024, emphasize the very low risk of clade I mpox importation to the US due to limited travel from affected regions and lack of direct commercial flights. This geographical and travel-related barrier significantly reduces the immediate threat to the US general population.

Secondly, the US government has demonstrated extensive preparedness measures, including enhanced surveillance systems, expanded diagnostic testing capacity, and substantial vaccine distribution. These measures, outlined in the CDC's August 22, 2024 media release, indicate a high level of readiness to detect and manage potential clade I cases, which likely increases the CDC's confidence in maintaining the current risk assessment.

While global developments such as the WHO's declaration of a Public Health Emergency of International Concern and the spread of clade Ib to new countries are concerning, the CDC's risk assessment for the US general population is primarily based on US-specific factors. The agency's recent communications consistently emphasize the very low risk to most Americans, suggesting a high bar for changing the risk assessment.

However, the situation remains dynamic, and there are factors that could potentially lead to a reassessment. The spread of clade Ib outside Africa, albeit limited, and uncertainties about vaccine effectiveness against the new strain introduce elements of unpredictability. Additionally, the WHO's emergency declaration could pressure the CDC to reassess its stance. These factors, combined with the time remaining until October 1, 2024, leave a small but non-zero probability of a risk level change.

Final prediction: 3.0%

Outcome: Risk level declared as LOW.

Key factors influencing my modified inside prediction:

Factor 1: Catalent's explicit statement on August 30, 2024, about being unable to file its annual report on time. Reflection: This factor is highly significant and the most recent direct statement from the company regarding its filing status. It comes from the most credible source possible - the company itself. This information strongly indicates that Catalent is likely to face challenges with its 2024 10-K filing. The recency and credibility of the source make this factor very influential in predicting the likelihood of timely filing. A possible counter-argument could be that the company might have overstated its challenges to manage expectations and could potentially resolve issues faster than anticipated.

Factor 2: The ongoing complexities introduced by the pending Novo Holdings merger, announced on February 5, 2024. Reflection: This factor is significant as major corporate transactions often impact financial reporting timelines. The information comes from official company announcements and is therefore credible. While not as recent as Factor 1, it provides important context for the company's current situation. It could directly influence the filing timeline due to the additional complexities and resources required for the merger process. A counter-argument could be that the pending merger might incentivize the company to expedite its reporting to facilitate the transaction.

Factor 3: Catalent's history of financial reporting issues, including the NYSE notice in September 2023 for not complying with timely filing criteria for its 2023 10-K. Reflection: This factor establishes a pattern of reporting difficulties, which adds weight to the likelihood of future delays. The information comes from credible sources (NYSE notices and company disclosures). While not as recent as Factors 1 and 2, it provides important historical context. It suggests a systemic issue rather than a one-time problem, strengthening the case for potential delays. A counter-argument could be that past issues might have spurred the company to improve its reporting processes.

Factor 4: Significant financial challenges reported for fiscal year 2024, including net loss and goodwill impairment charges. Reflection: This factor is significant as financial difficulties can complicate the reporting process. The information comes from recent company financial reports and is therefore credible. It provides context for the company's overall situation and the challenges it faces in preparing accurate financial statements. However, its direct impact on the filing timeline is less clear than Factors 1-3. A counter-argument could be that these challenges were known and accounted for in the company's planning process.

New factors from web search results:

Factor 5: Catalent's Q3 2024 financial results reported on May 30, 2024, showing improvement in revenue and adjusted EBITDA. Reflection: This factor provides the most recent financial performance data available. It comes from an official company report and is therefore highly credible. The improved performance could potentially facilitate easier financial reporting. However, it does not directly address the company's ability to file its annual report on time. There's a risk of overreacting to this positive news, as improved financial performance doesn't necessarily translate to improved reporting capabilities.

Factor 6: The approval of Catalent's merger with Novo Holdings by stockholders, announced on June 13, 2024. Reflection: This recent development confirms the progression of the merger process. It comes from an official company announcement and is highly credible. While it doesn't directly impact the filing timeline, it could potentially add complexity to the reporting process or, conversely, provide additional resources and motivation to complete filings promptly. There's a risk of overestimating the impact of this factor on the specific question of timely 10-K filing.

Reflection:

Upon careful reflection, I recognize a potential weakness in my analysis: I may be giving too much weight to Catalent's past reporting issues and not fully considering the potential positive impact of the pending merger with Novo Holdings. While historical patterns are important, the resources and expertise of Novo Holdings could potentially help expedite the reporting process. Additionally, I may be underestimating the company's motivation to file on time given the pending merger and improved Q3 results.

However, I don't believe these weaknesses are significant enough to substantially shift my prediction. The explicit statement from Catalent on August 30, 2024, about being unable to file its annual report on time remains the most recent and direct piece of information relevant to the question.

Reminder about question and resolution criteria: The question asks: "Will Catalent, Inc., file its 10-K annual report with the SEC before October 1, 2024?" The resolution criteria specify that a Form 10-K must be filed for the fiscal year ended June 30, 2024, at the specified SEC Edgar link before October 1, 2024, for a "Yes" resolution. Any other outcome, including inaccessibility of the SEC Edgar site after 7 days, results in a "No" resolution.

Justification: My final prediction of a 15% chance that Catalent will file its 10-K annual report with the SEC before October 1, 2024, is primarily driven by the company's explicit statement on August 30, 2024, about being unable to file its annual report on time. This recent and direct communication from the company carries significant weight in the prediction.

The ongoing merger process with Novo Holdings, while potentially providing additional resources, also introduces complexities that could further complicate the timely filing of the 10-K. Historical patterns of delayed filings, including the NYSE notice in September 2023 for non-compliance with timely filing criteria, suggest systemic challenges in Catalent's financial reporting processes.

While the improved Q3 2024 financial results and the stockholder approval of the Novo Holdings merger are positive developments, they do not directly address the company's ability to overcome the filing challenges explicitly stated on August 30. The short timeframe between this statement and the October 1 deadline further reduces the likelihood of a timely filing.

The 15% probability acknowledges the small chance that Catalent might overcome these challenges more quickly than anticipated, perhaps due to increased urgency related to the pending merger or unforeseen improvements in their reporting processes. However, given the explicit nature of their recent statement and the short timeframe for resolution, a higher probability is not warranted based on the available information.

Final prediction: 15.0%

Outcome: Report filed on September 6.

Here is my analysis of the key factors influencing my modified inside prediction after discussion with my colleague:

Factor 1: Taylor Swift's past endorsement of Biden-Harris in 2020 This factor remains significant as it establishes a precedent for Swift's political involvement and alignment with the Democratic party. It demonstrates her willingness to support Harris as part of a presidential ticket. The information comes from a credible source (CBS News) but is from 4 years ago, making it somewhat dated. While it suggests a likelihood of supporting Harris again, it doesn't guarantee a future endorsement. A counter-argument is that political landscapes and personal views can change significantly in 4 years.

Factor 2: Lack of current endorsement from Swift for the 2024 election This factor is crucial as it directly relates to the question at hand. The lack of endorsement as of August 30, 2024, could indicate hesitation or a deliberate choice to stay out of politics this election cycle. However, it's also possible that Swift is waiting for a strategic moment to make an endorsement. The recency of this information (August 30, 2024) makes it highly relevant to our prediction. This factor is based on the absence of information from multiple credible sources and is not white noise. A counter-argument is that absence of evidence is not evidence of absence - Swift could be planning an endorsement but keeping it private.

Factor 3: "Swifties for Kamala" fan group activities This factor demonstrates strong support for Harris among Swift's fanbase. The group has raised $142,000 for Harris as of August 30, 2024, with $122,000 collected during a Zoom event on August 29, 2024. About 34,000 people joined the call, which featured appearances from notable political figures. This information comes from multiple credible sources (NBC News, Variety) and is very recent. While it doesn't directly predict Swift's actions, it provides context for the potential impact of her endorsement. A counter-argument is that fan actions don't necessarily reflect or influence the artist's views.

Factor 4: Swift's increased political engagement since 2018 This factor provides context for Swift's political evolution, suggesting she has become more comfortable using her platform for political purposes over time. However, it doesn't necessarily predict her actions for the 2024 election. This information comes from multiple sources over time and is more of a trend than a specific recent event. A counter-argument is that past behavior doesn't always predict future actions, especially in the rapidly changing landscape of celebrity political involvement.

Factor 5: Swift's current tour schedule This factor, highlighted by my colleague, indicates Swift's commitments during the period in question. She has tour dates scheduled through August 2024. While it doesn't preclude an endorsement, it suggests that Swift's focus may be on her tour rather than political activities. This information comes from credible sources reporting on Swift's tour schedule and is current. A counter-argument is that celebrities often make political statements while on tour, using their platform to reach large audiences.

Examining the new web information queried at the end of step 4, I find the following significant new factors:

Factor 6: Swift's non-partisan voter engagement message in March 2024 On March 5, 2024, Swift posted a brief, nonpartisan message on Instagram encouraging her 282 million followers to vote in the presidential primaries. She did not endorse any candidates. This information comes from a credible source (New York Times) and is relatively recent. It suggests Swift is still interested in political engagement but may be taking a more neutral stance in this election cycle. This could indicate a reluctance to make a partisan endorsement, but it's also possible she's building up to a more specific statement closer to the election.

Factor 7: Increased speculation and interest in Swift's potential endorsement Multiple recent news sources (NPR, Virginia Tech News) report on the widespread speculation about Swift's potential endorsement. While this doesn't directly influence Swift's decision, it indicates the perceived importance of her potential endorsement and could potentially create pressure for Swift to make a statement. This information is very recent and comes from credible sources. However, it could also be considered "white noise" as it doesn't provide concrete evidence of Swift's intentions.

Factor 8: Swift's 2018 endorsement of Democratic candidates in Tennessee In October 2018, Swift made her first political endorsement, supporting Democratic candidates in Tennessee. This information comes from a credible source (Swift's own Instagram post) but is quite dated. It provides context for Swift's political engagement but may not be directly relevant to her current stance. A counter-argument is that this past behavior occurred under different political circumstances and may not reflect her current views.

Reflecting on my analysis, I recognize that I may be overweighting the significance of the "Swifties for Kamala" group's activities. While it demonstrates enthusiasm among Swift's fans, it doesn't necessarily reflect Swift's own intentions. I may also be underestimating the importance of Swift's current tour schedule as a potential barrier to political engagement.

I also note that I haven't fully considered the potential strategic reasons for Swift to delay an endorsement. Given her past pattern of making political statements closer to election dates, the September timeframe specified in the question may actually decrease the likelihood of an endorsement.

Reminder about question and resolution criteria: The question asks whether Taylor Swift will publicly endorse Kamala Harris for president before October 1, 2024. The endorsement must be a clear, unambiguous statement of support for Harris's presidential campaign, coming directly from Swift or her official representatives. Vague statements or implications of support will not count. The endorsement must occur before October 1, 2024, 12:00 AM EDT.

Justification: Based on the analysis of multiple factors, I believe there is a low probability that Taylor Swift will publicly endorse Kamala Harris for president before October 1, 2024. While Swift has a history of political engagement and previously supported the Biden-Harris ticket in 2020, several factors suggest she may not make an endorsement within the specified timeframe.

Firstly, Swift's most recent political engagement (as of March 2024) was nonpartisan, focusing on encouraging voter participation rather than endorsing specific candidates. This could indicate a shift towards a more neutral stance in this election cycle. Additionally, Swift's busy tour schedule through August 2024 may limit her involvement in political activities, although it doesn't preclude an endorsement entirely.

The activities of the "Swifties for Kamala" group, while demonstrating enthusiasm among Swift's fanbase, do not necessarily reflect Swift's own intentions. In fact, the group's success in fundraising and mobilizing support for Harris could potentially reduce the perceived need for Swift to make an official endorsement.

Historical patterns suggest that if Swift were to make an endorsement, it might come closer to the election date. The October 1 deadline specified in the question may be too early based on her past behavior. Furthermore, the intense speculation and interest in Swift's potential endorsement could make her more cautious about making a public statement, as she may want to avoid being seen as succumbing to public pressure.

However, it's important to note that Swift has become increasingly politically engaged since 2018, and she has previously supported Harris as part of the Biden-Harris ticket. The possibility of an endorsement cannot be ruled out entirely, especially given Swift's influence and the high stakes of the presidential election.

Final prediction: 12.5%

Outcome: Taylor Swift endorsed Harris after numerous AI imitations of Swift endorsing Trump circulated on the internet.

Key factors influencing my modified inside prediction:

Factor 1: Record-breaking 23 Emmy nominations for "The Bear" Season 2 This factor is highly significant for the resolution of the question. The record-breaking nominations demonstrate exceptional industry recognition and recent success. This information is very recent (from the 2024 Emmy nominations) and comes from credible sources reporting on official Emmy announcements. It directly influences the question by showing The Bear's strong standing among Emmy voters. A possible counter-argument could be that nominations don't always translate to wins, especially in the top category.

Factor 2: Previous win for Outstanding Comedy Series in 2023 This factor is also highly significant. The Bear's win last year establishes it as the incumbent favorite. This information is recent (from the 2023 Emmy Awards) and comes from official Emmy results. It directly influences the question by showing The Bear has already been recognized as the best comedy series by Emmy voters. A counter-argument could be that repeat wins are not guaranteed, and voters might want to recognize a different show this year.

Factor 3: Critical reception of Season 3 The critical reception for The Bear's latest season is important, as it often correlates with Emmy success. This information is very recent (June 2024) and comes from reputable sources like The Hollywood Reporter. While not as directly influential as the nomination count or previous win, it provides context for the show's quality and industry perception. A counter-argument could be that Emmy voters might not have had time to fully watch and digest Season 3 before voting.

Factor 4: Strong competition from other nominees Understanding the strength of other nominees like "Abbott Elementary" and "Hacks" is crucial for contextualizing The Bear's chances. This information is recent and from credible sources reporting on Emmy nominations. It doesn't directly influence The Bear's chances but provides necessary context for assessing its relative position. A counter-argument could be that the strength of competition is subjective and might not accurately reflect voter preferences.

New factors from web search results:

Factor 5: Mixed critical reception of Season 3 The Hollywood Reporter article from September 5, 2024, indicates that while Season 3 of The Bear has received generally positive reviews, some critics found it less compelling than previous seasons. This recent information from a credible source provides a more nuanced view of the show's critical reception. It could influence voter perception, but it's important not to overreact to a single article. This factor somewhat contradicts the overall positive reception noted earlier and adds complexity to the assessment.

Factor 6: Debate over The Bear's categorization as a comedy Multiple sources, including a Gold Derby article from 2023, question whether The Bear should be classified as a comedy for awards purposes. While this debate isn't new, its persistence could influence voter perception. This factor doesn't directly relate to the show's quality but could impact how voters approach their decision. It's important not to overreact to this factor, as the show has already won in the comedy category.

Factor 7: Ongoing industry labor issues The NPR article from July 2023 mentions the impact of industry strikes on Emmy campaigning. While this information is older, ongoing labor issues in the industry could still affect Emmy voting and the ceremony itself. This factor adds uncertainty to the prediction but doesn't directly relate to The Bear's quality or chances of winning.

Reflection: Upon reflection, I may have slightly overemphasized the importance of Season 3's critical reception. While it's relevant, Emmy voters are primarily considering Season 2, which received the record-breaking nominations. I should be cautious about letting the mixed reviews of Season 3 unduly influence my prediction.

I may also be underestimating the potential impact of the debate over The Bear's comedy categorization. While it hasn't prevented the show from winning before, persistent questions about genre could influence some voters, especially if they feel other nominees better represent the comedy category.

However, these potential weaknesses in my analysis don't significantly alter my overall assessment. The record-breaking nominations and previous win remain the most substantial factors, and they strongly support a high probability of The Bear winning again.

Reminder about question and resolution criteria: The question asks whether The Bear will win the 2024 Primetime Emmy Award for Outstanding Comedy Series. The resolution criteria specify that the show must win this specific award at the ceremony scheduled for September 15, 2024. My analysis and prediction must focus on this particular outcome.

Justification: The Bear enters the 2024 Emmy race as the clear frontrunner for Outstanding Comedy Series. Its record-breaking 23 nominations for Season 2 demonstrate exceptional industry recognition and support, surpassing even its strong showing in the previous year. This level of acclaim across multiple categories suggests broad and deep appreciation for the show among Emmy voters.

Furthermore, The Bear's win in this category at the 2023 Emmys establishes it as the incumbent favorite. While repeat wins are not guaranteed, they are common in the Outstanding Comedy Series category, especially for shows that maintain or increase their quality and recognition. The fact that The Bear has garnered even more nominations this year than last suggests it has not only maintained but potentially increased its standing among voters.

The critical reception for Season 2, which is the primary season under consideration for this award, was overwhelmingly positive. While there have been some mixed reviews for Season 3, these are less relevant to the voting for this award and are unlikely to significantly impact The Bear's chances. The show's ability to consistently deliver high-quality content across multiple seasons strengthens its position as a deserving winner.

Competition from other nominees, particularly established shows like "Abbott Elementary" and "Hacks," should not be discounted. However, given The Bear's previous win and increased nominations, it appears to have a substantial advantage over its competitors. The ongoing debate about whether The Bear truly fits the comedy category could potentially influence some voters, but this hasn't prevented the show from succeeding in the category before.

Considering all these factors, while acknowledging the inherent uncertainty in any awards prediction, The Bear appears to have a very strong chance of winning the 2024 Primetime Emmy Award for Outstanding Comedy Series.

Final prediction: 84.7%

Outcome: Hacks won. The system performs especially poorly when asked about the outcome of winning competitions. There are usually multiple nominees and the system seems to favor the nominee in question.

These are my step by step prompts:

step_1 = """
<instructions>
You’re a superforecaster analyzing the following question:
    <question>
    {question}
    {resolution_criteria}
    </question>
Today’s date is {current_date}. 
    You are currently on step 1 of your problem-solving process. Complete the analysis required for ONLY step 1 of your process. Think about and re-read the question in a slow, stepwise manner. Do some initial fermi-izing. Mirror your thinking after many of your previous analyses and guiding principles. Then, think about what you would like to know more about to have a better chance at nailing the forecasting question. Format your answer as below.
Thinking:
{{ Insert your step-by-step thought process about the following fields. Do not provide any answers yet, you are still thinking. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}
Question interpretation:
{{ Insert your objective interpretation of the question here }}
Fermi-analysis:
{{ Insert your fermi analysis here. Split it into analysis you would use to establish the outside and inside view respectively. }}
Relevant information:
{{ Insert the specific areas you would like to know more about here }}
Make this answer sufficiently detailed that even if you came back to it a few months later to determine if your prediction would have to be revised, you are able to quickly get a detailed overview of your analysis. As such, retain all important information.
If there’s anything you’d like to search up for step 2 to establish an outside view, please append a list of up to four search queries to your analysis in the following format, ensuring it begins with 'Search queries: ', including the source (Google or Google News) with each query:
        Search queries:
        1. [Query details] (Google)
        2. [Query details] (Google News)
Keep search queries short and carefully phrase each search query to maximize the likelihood of finding the most relevant web content. Make sure to focus these queries all towards establishing an outside view, referring specifically to the fermi analysis for the outside view you conducted.
</instructions>
"""

step_2= """
<instructions>
You’re a superforecaster analyzing the following question:
    <question>
    {question}
    {resolution_criteria}
    </question>
Today’s date is {current_date}. Consideration of the timeframe from today till the date the question is resolved might be important, depending on the context of the question.
    You are currently on step 2 of your problem-solving process. You can find the information you have gathered so far together with your analysis from step 1 in the <report> XML tag below. Ignore irrelevant advertisements and junk information from web search results.
<report>
{information}
</report>
Your first task is to analyze your step 1 findings and corresponding search results. What was your objective interpretation of the question? Did you make sure to avoid common pitfalls like the bait-and-switch error? How did you Fermi-ize the question at hand? What are the tractable sub-problems you investigated? What questions did you pose to resolve these sub-problems? Then, examine the search results, identify and write down exact answers to all these questions. Make these answers as specific as possible. Retain key information like numbers (count, percentage, proportion, basically anything numerical!), dates of authorship and source of information. These answers will be important for establishing a base rate. Format your answers as below, sticking to paragraph form over point form.
Step 1 report dissection:
{{ Insert your reflection here }}
Questions posed and search result examination:
{{ List each question posed in the step 1 analysis and provide a detailed answers in full paragraph form extracted from search results }}
Timeframe until question resolution:
{{Analyze the time frame from today till the question is resolved? Is it significant to the establishment of an outside view? Why or why not? }}
Next, complete the analysis required ONLY for step 2. You may be tempted to jump into the details that would only pertain to the inside view, but remember you are establishing an outside view. Base it as far as possible on historical trends and similar past cases with little attention to exact current circumstances about the forecasting quesiton, unless absolutely vital. Provide your justification and fermi analysis if relevant in paragraph form and establish your outside view.
Thinking:
{{ Insert your step-by-step thought process for arriving at an outside view. Do not provide an outside view just yet, you are still thinking. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework. Reflect on whether you might be placing too much emphasis on current circumstances.}}
When crafting an outside view, you know not to treat 0.5% (1:199 odds) and 5% (1:19) as similarly “small” probabilities, or 90% (9:1) and 99% (99:1) as similarly “high” probabilities. As the odds show, they are markedly different, so you output your probabilities accordingly.
Before finalizing your outside view:
1. Explicitly state the historical base rate you're using and justify its relevance.
2. Identify at least three historical examples most similar to the current situation and explain key similarities and differences.
3. List any cognitive biases you might be susceptible to and explain how you're mitigating them.
4. Perform a brief sensitivity analysis: how would your prediction change if your key assumptions were slightly off?
5. Identify any major points of potential expert disagreement on this question.
6. Clearly state your reference class and justify its selection.
Justification:
{{ Insert your justification here, referencing past trends used to establish the outside view. You might want to write about a paragraph of justification. Reflect on whether you might be placing too much emphasis on current circumstances. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.  }}
Outside view:
{{ Make your outside view prediction here to a 1% significance. }}
Make this answer sufficiently detailed that even if you came back to it a few months later to determine if your prediction would have to be revised, you are able to quickly get a detailed overview of your analysis. As such, retain all important information, including direct quotes from the information sources gathered so far.
Once the inside view is established, you will likely need to refine this base rate even further by means of further web search results in step 3. For this purpose, please append a list of up to three search queries to your analysis in the following format, ensuring it begins with 'Search queries: ', including the source (Google or Google News) with each query:
        Search queries:
        1. [Query details] (Google)
        2. [Query details] (Google News)
Keep search queries short and carefully phrase each search query to maximize the likelihood of finding the most relevant web content. You will likely want to search up the most relevant current information specific to the question here.
In addition to web searches, knowledge of current breaking news will be invaluable for an inside view as well. As such, append ONE additional search query that will be used to retrieve relevant breaking news articles. You want to have access to a broad but relevant range of breaking news articles pertaining to the question. Phrase your search query carefully as such, ensuring it isn’t overly specific. Add it on in the format below, setting Assistant as the source:
    4.  [Query details] (Assistant)
Finally, you will be able to submit a final search query to the perplexity API for further research purposes. As perplexity uses large-langauge model based processing, your search query can be longer and more nuanced. Add the final query as below:
    5.  [Query details] (Perplexity)
</instructions>

"""

step_3 = """

<instructions>
You're a superforecaster analyzing the following question:
    <question>
    {question}
    {resolution_criteria}
    </question>
Today's date is {current_date}. It is crucial to consider the timeframe from today to the day the question is resolved.

You are currently on step 3 of your problem-solving process. You can find the information you have gathered so far together with your analysis from step 2 in the <report> XML tag below. Ignore irrelevant advertisements and junk information from web search results.

<report>
{information}
</report>

Your first task is to analyze your step 2 findings and corresponding search results. Follow these steps:

1. Explicitly state the outside view probability you established in step 2, including the exact numerical value.
2. Summarize your Fermi analysis of the question thus far.
3. List the search queries you posed to find current inside information relevant to the question.
4. Examine the search results, identify and write down exact answers to all these questions. Make these answers as specific as possible. Retain key information like numbers (count, percentage, proportion, basically anything numerical!), dates of authorship and source of information. These answers will be important for establishing an inside view.

Format your answers as below, sticking to paragraph form over point form:

Thinking:
{{ Insert your thought process about dissecting your step 2 report. Think about the most prudent areas of discussion to focus on as well as more minor details you might potentially miss out in your report discussion below.}}

Step 2 report dissection:
{{ Insert your reflection here, including the explicit statement of your outside view probability }}

Questions posed and search result examination:
{{ List each question posed in the step 2 analysis and provide detailed answers in full paragraph form extracted from search results. Include the date of the information and its source. }}

Next, complete the analysis required ONLY for step 3. Establish your inside view. Follow the following format of analysis:

Factors influencing inside view:
{{Factor 1: Brief details}}
{{Factor 1: A reflection on the significance of this factor towards the resolution of the question. Consider:
- Does it contradict previous factors considered?
- What is its recency relative to other factors?
- Is it from a credible source?
- Could it just be noise?
- How strongly can it directly influence the resolution of the question?
- How does this factor compare to historical trends or base rates? }}

Repeat for the other factors.

When crafting an inside view, remember that probabilities that may seem similarly "small" or "large" can be markedly different. For example, 0.5% (1:199 odds) and 5% (1:19) are very different, as are 90% (9:1) and 99% (99:1). Output your probabilities accordingly.

Thinking:
{{ Insert your step-by-step thought process for arriving at an inside view. Do not provide an inside view yet, you are still thinking. Consider:
- Whether the question has multiple delineated possible outcomes, even if they're grouped into two categories for forecasting purposes.
- That many scenarios have a range of distinct results, rather than just "it happens" or "it doesn't."
- How the timeframe until resolution might affect your analysis.
- Potential biases in your reasoning, such as recency bias or overconfidence.
- How your inside view compares to your outside view, and justify any significant differences.
Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}

Justification:
{{Insert your justification here, providing a paragraph-style justification. Consider:
- All factors holistically, with the dragonfly-eye thinking approach.
- Whether you may be over- or under-reacting to the information shaping your inside view.
- The possibility that your forecast is in the wrong direction. What would the outcome likely be and what would be the most influential factors in your post-mortem analysis?
- How your justification aligns with both historical trends and current data.
Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}

Inside view:
{{Formulate an inside view, keeping your answer to 1% significance. Ensure that your inside view probability is logically consistent with your outside view and just


"""

step_4 = """
<instructions>
You’re a superforecaster analyzing the following question:
    <question>
    {question}
    {resolution_criteria}
    </question>
Today’s date is {current_date}. Consideration of the timeframe from today till the date the question is resolved might be important, depending on the context of the question.

    You are currently on step 4 of your problem-solving process. You can find the information you have gathered so far together with your analysis from step 3 in the <report> XML tag below. Ignore irrelevant advertisements and junk information from web search results.
<report>
{information}
</report>
You are working with one colleague forecaster. Their view is provided below.
    <colleague_view>
    {colleague_views}
    </colleague_view>
Your first task is to analyze your step 3 findings and corresponding inside prediction made. What are the factors influencing the prediction you made and why? Then, examine your colleague’s analysis to identify the key factors influencing the prediction they made as well as their initial inside prediction. 
During this analysis, be objective, trying not to favour your own or your colleague’s views. Avoid under- or over- confidence biases. Format your analysis as below:
The key factors you presented:
{{Factor 1: Brief details}}
{{Factor 1: A reflection on the significance of this factor towards the resolution of the question. What is its recency relative to other factors? Is it from a credible source? Could it just be white noise? How strongly can it directly influence the resolution of the question? }}
Repeat for the other factors.
The key factors your colleague presented:
{{Factor 1: Brief details}}
{{Factor 1: A reflection on the significance of this factor towards the resolution of the question. Does it contradict other factors considered? Did you consider this factor as well? What is its recency relative to other factors? Is it from a credible source? Could it just be white noise? How strongly can it directly influence the resolution of the question? }}
Then, determine how your initial inside view prediction should be adjusted:
Thinking:
{{ Insert your step-by-step thought process for arriving at an updated prediction. Do not provide an updated prediction just yet, you are still thinking. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}
When updating your prediction, you know not to treat 0.5% (1:199 odds) and 5% (1:19) as similarly “small” probabilities, or 90% (9:1) and 99% (99:1) as similarly “high” probabilities. As the odds show, they are markedly different, so you output your probabilities accordingly.
Updated prediction and justification:
{{ Insert your updated prediction and a paragraph-style justification here. Carefully consider all factors holistically, with the dragonfly-eye thinking approach. Reflect on whether you may be over or under reacting to your colleague’s views. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}
Make your answers sufficiently detailed that even if you came back to it a few months later to determine if your prediction would have to be revised, you are able to quickly get a detailed overview of your analysis. As such, retain all important information, including direct quotes from the information sources gathered so far, and keep your reasoning as transparent/detailed as possible.
Finally, you might still be left with some uncertainty and unanswered questions you would like to know more about before making a final prediction. You also want to be careful, keeping an open mind to alternative perspectives you might not have yet considered. For this purpose, please append to your analysis a list of four search queries you would like to get web information from in the following format, ensuring it begins with 'Search queries: ', including the source (Google or Google News) with each query:
        Search queries:
        1. [Query details] (Google)
        2. [Query details] (Google News)
Keep search queries short and carefully phrase each search query to maximize the likelihood of finding the most relevant web content. Use these queries wisely.
</instructions>

"""

step_5 = """
<instructions>
You’re a superforecaster analyzing the following question:
    <question>
    {question}
    {resolution_criteria}
    </question>
Today’s date is {current_date}. Consideration of the timeframe from today till the date the question is resolved might be important, depending on the context of the question.
    You are currently on step 5 of your problem-solving process. You can find the information you have gathered so far together with your analysis from step 4 in the <report> XML tag below. Ignore irrelevant advertisements and junk information from web search results.
<report>
{information}
</report>
Your first task is to analyze your step 5 analysis and corresponding shifts to your inside prediction made after discussion with your colleague. What is your new shifted inside prediction numerically and why'd you choose to shift it? What are the key factors influencing your modified inside prediction? For each factor, explicitly include as much specific information as possible, including numbers, timelines, publication dates and source of information. Use the format below:
{{Factor 1: Brief details}}
{{Factor 1: A reflection on the significance of this factor towards the resolution of the question. What is its recency relative to other factors? Is it from a credible source? Could it just be white noise? How strongly can it directly influence the resolution of the question? What would be a possible counter-argument to your stance on the significance of this factor?}}
Repeat for the other factors.
Then, carefully examine the web information you queried for at the end of step 4. Do the search results present any new or conflicting factors? Are there any new factors presented that would strengthen your stance on the forecasting question? What is the significance of these factors? Analyze using the format below:
{{Factor 1: Brief details}}
{{Factor 1: A reflection on the significance of this factor towards the resolution of the question. Does it contradict other factors considered? What is its recency relative to other factors? Is it from a credible source? Could it just be white noise? How strongly can it directly influence the resolution of the question? Are you possibly over- or under- reacting to this factor?}}
Repeat for the other factors.
Write all details in paragraph format. Carefully evaluate your analysis thus far. Use the insights you gathered from your decades experience as a superforecaster as a backdrop.
Reflect on whether you’ve fell victim to intellectual or psychological fallacies you’ve encountered and documented over your years of superforecasting. Now, be your own devil’s advocate. Reflect on your entire thought process thus far, identifying all holes and lapses. This does not necessarily mean shifting your prediction to a less extreme. Heck, it is even possible your prediction could be more extreme. It is even possible that your analysis does not have any significant fallacies. The point, however, is to, from a rational viewpoint, evaluate your thinking thus far to identify any weaknesses—from putting too little faith in a new bit of information to assuming a new bit of information actually affects the resolution of the question. You may refer to past errors you documented of your colleagues and yourself for examples to write this analysis. 
Reflection:
{{ Insert your reflection here. Are any of your identified weaknesses sufficiently significant to warrant a shift to your prediction? }}
Reminder about question and resolution criteria:
{{Remind yourself of the question and resolution criteria. Be sure that your analysis is fully in-line with the requirements stated in the question and resolution criteria.}}
Thinking:
{{ Insert your step-by-step thought process for arriving at a final prediction. Do not provide a final prediction yet, you are still thinking. Hold your reasoning standards to the Paul-Elder Critical Thinking Framework.}}
When making your final prediction, you know not to treat 0.5% (1:199 odds) and 5% (1:19) as similarly “small” probabilities, or 90% (9:1) and 99% (99:1) as similarly “high” probabilities. As the odds show, they are markedly different, so you output your probabilities accordingly.
Then, provide your final prediction together with a paragraph-styled justification. Format as below. Recall all your previous experience superforecasting and the style of your past forecasts. 
Justification:
{{ Insert your justification here. You might want to write 3-4 paragraphs of justification. Carefully consider all factors holistically, with the dragonfly-eye thinking approach. As a good critical thinking practice, consider the possibility that your forecast is in the wrong direction. What would the outcome of the forecasting question likely be and what would be the most influential factors in your post-mortem analysis? Should this affect your final prediction? Hold your reasoning standards to the Paul-Elder Critical Thinking Framework. }}
Final prediction:
{{ Your final prediction, to a precision of 0.1%. No other commentary is needed here. }}

</instructions>

"""

I need suggestions about how these prompts can be better fine-tuned to optimize accuracy and correctness in forecasts. A suggestion regarding model temperature would be appreciated too (I am using Claude-3.5-sonnet at a 0.4 temperature).