Custom prompts for chain types that aren't "stuff" in RetrievalQA

### Feature request

Consider the following example:
```python
# All the dependencies being used
import openai
import os
from dotenv import load_dotenv

from langchain.chains import RetrievalQA
from langchain.document_loaders import TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.chat_models import ChatOpenAI
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.prompts import PromptTemplate

load_dotenv()

openai.organization = os.getenv("OPENAI_ORG_ID_")
openai.api_key = os.getenv("OPENAI_API_KEY")

# Load up a text file
loader = TextLoader("foo.txt")
documents = loader.load()

# Split text into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(documents)

# Set up chroma 
embeddings = OpenAIEmbeddings()
docsearch = Chroma.from_documents(texts, embeddings)

# I want a custom prompt that asks for the output in JSON format
prompt_template = """
Use the following pieces of context to answer the question at the end. 
If you don't know the answer, output 'N/A', don't try to 
make up an answer.

{context}

Question: {question}
Answer in JSON format:
"""

PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

llm = ChatOpenAI(model_name="gpt-3.5", temperature=0)

# This is what's done in the Python docs
chain_type_kwargs = {'prompt': PROMPT}
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=docsearch.as_retriever(), chain_type_kwargs=chain_type_kwargs)

query = "Foo bar"
res = qa.run(query)
```
If we use anything other than `"stuff"` for the `chain_type` parameter in `RetrievalQA.from_chain_type`, we'll get the following error from that line:

```terminal
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/retrieval_qa/base.py", line 91, in from_chain_type
    combine_documents_chain = load_qa_chain(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/question_answering/__init__.py", line 238, in load_qa_chain
    return loader_mapping[chain_type](
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/chains/question_answering/__init__.py", line 196, in _load_refine_chain
    return RefineDocumentsChain(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/load/serializable.py", line 61, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for RefineDocumentsChain
prompt
  extra fields not permitted (type=value_error.extra)
```
Is there anything in particular that prevents custom prompts being used for different chain types? Am I missing something? Open to any help and/or guidance.

### Motivation

I'm trying to perform QA on a large block of text and so using map_reduce or refine is preferable over stuff. I also want to perform the QA with a custom prompt as I need the chain's output to be in JSON format for parsing. When using stuff for text that doesn't surpass the token limit, it works as expected.

### Your contribution

Happy to contribute via a PR if someone identifies that what I'm suggesting isn't impossible.