Package: carbon-python-sdk
CarbonConnect external data to LLMs, no matter the source.Table of ContentsRequirementsInstallationGetting StartedAsyncRaw HTTP ResponseReferencecarbon.auth.get_access_tokencarbon.auth.get_white_labelingcarbon.data_sources.query_user_data_sourcescarbon.data_sources.revoke_access_tokencarbon.embeddings.get_documentscarbon.embeddings.get_embeddings_and_chunkscarbon.embeddings.upload_chunks_and_embeddingscarbon.files.create_user_file_tagscarbon.files.deletecarbon.files.delete_file_tagscarbon.files.delete_manycarbon.files.get_parsed_filecarbon.files.get_raw_filecarbon.files.query_user_filescarbon.files.query_user_files_deprecatedcarbon.files.resynccarbon.files.uploadcarbon.files.upload_from_urlcarbon.files.upload_textcarbon.health.checkcarbon.integrations.connect_freshdeskcarbon.integrations.create_aws_iam_usercarbon.integrations.get_oauth_urlcarbon.integrations.list_confluence_pagescarbon.integrations.list_data_source_itemscarbon.integrations.list_folderscarbon.integrations.list_labelscarbon.integrations.sync_confluencecarbon.integrations.sync_data_source_itemscarbon.integrations.sync_filescarbon.integrations.sync_gmailcarbon.integrations.sync_outlookcarbon.integrations.sync_rss_feedcarbon.integrations.sync_s3_filescarbon.organizations.getcarbon.users.getcarbon.users.toggle_user_featurescarbon.utilities.fetch_urlscarbon.utilities.fetch_youtube_transcriptscarbon.utilities.process_sitemapcarbon.utilities.scrape_sitemapcarbon.utilities.scrape_webcarbon.utilities.search_urlscarbon.webhooks.add_urlcarbon.webhooks.delete_urlcarbon.webhooks.urlsRequirementsPython >=3.7Installationpipinstallcarbon-python-sdk==0.1.1Getting StartedfromcarbonimportCarbon# Generally this is done in the backend to avoid exposing API key to the clientcarbon=Carbon(api_key="YOUR_API_KEY",customer_id="YOUR_CUSTOMER_ID",)token=carbon.auth.get_access_token()# Once an access token is obtained, it can be passed to the frontend# and used to instantiate the SDK client without an API keycarbon=Carbon(access_token=token.access_token)# use SDK as usualwhite_labeling=carbon.auth.get_white_labeling()# etc.Asyncasyncsupport is available by prependingato any method.importasynciofrompprintimportpprintfromcarbonimportCarbon,ApiExceptioncarbon=Carbon(access_token="YOUR_API_KEY",api_key="YOUR_API_KEY",customer_id="YOUR_API_KEY",)asyncdefmain():try:# Get Access Tokenget_access_token_response=awaitcarbon.auth.aget_access_token()print(get_access_token_response)exceptApiExceptionase:print("Exception when calling AuthApi.get_access_token:%s\n"%e)pprint(e.body)ife.status==422:pprint(e.body["detail"])pprint(e.headers)pprint(e.status)pprint(e.reason)pprint(e.round_trip_time)asyncio.run(main())Raw HTTP ResponseTo access raw HTTP response values, use the.rawnamespace.frompprintimportpprintfromcarbonimportCarbon,ApiExceptioncarbon=Carbon(access_token="YOUR_API_KEY",api_key="YOUR_API_KEY",customer_id="YOUR_API_KEY",)try:# Get Access Tokenget_access_token_response=carbon.auth.raw.get_access_token()pprint(get_access_token_response.body)pprint(get_access_token_response.body["access_token"])pprint(get_access_token_response.body["refresh_token"])pprint(get_access_token_response.headers)pprint(get_access_token_response.status)pprint(get_access_token_response.round_trip_time)exceptApiExceptionase:print("Exception when calling AuthApi.get_access_token:%s\n"%e)pprint(e.body)ife.status==422:pprint(e.body["detail"])pprint(e.headers)pprint(e.status)pprint(e.reason)pprint(e.round_trip_time)Referencecarbon.auth.get_access_tokenGet Access Token🛠️ Usageget_access_token_response=carbon.auth.get_access_token()🔄 ReturnTokenResponse🌐 Endpoint/auth/v1/access_tokenget🔙Back to Table of Contentscarbon.auth.get_white_labelingReturns whether or not the organization is white labeled and which integrations are white labeled:param current_user: the current user
:param db: the database session
:return: a WhiteLabelingResponse🛠️ Usageget_white_labeling_response=carbon.auth.get_white_labeling()🔄 ReturnWhiteLabelingResponse🌐 Endpoint/auth/v1/white_labelingget🔙Back to Table of Contentscarbon.data_sources.query_user_data_sourcesUser Data Sources🛠️ Usagequery_user_data_sources_response=carbon.data_sources.query_user_data_sources(pagination={"limit":10,"offset":0,},order_by="created_at",order_dir="desc",filters={"source":"GOOGLE_DRIVE",},)⚙️ Parameterspagination:Paginationorder_by:OrganizationUserDataSourceOrderByColumnsorder_dir:OrderDirfilters:OrganizationUserDataSourceFilters⚙️ Request BodyOrganizationUserDataSourceQueryInput🔄 ReturnOrganizationUserDataSourceResponse🌐 Endpoint/user_data_sourcespost🔙Back to Table of Contentscarbon.data_sources.revoke_access_tokenRevoke Access Token🛠️ Usagerevoke_access_token_response=carbon.data_sources.revoke_access_token(data_source_id=1,)⚙️ Parametersdata_source_id:int⚙️ Request BodyRevokeAccessTokenInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/revoke_access_tokenpost🔙Back to Table of Contentscarbon.embeddings.get_documentsFor pre-filtering documents, usingtags_v2is preferred to usingtags(which is now deprecated). If bothtags_v2andtagsare specified,tagsis ignored.tags_v2enables
building complex filters through the use of "AND", "OR", and negation logic. Take the below input as an example:{"OR":[{"key":"subject","value":"holy-bible","negate":false},{"key":"person-of-interest","value":"jesus christ","negate":false},{"key":"genre","value":"religion","negate":true}{"AND":[{"key":"subject","value":"tao-te-ching","negate":false},{"key":"author","value":"lao-tzu","negate":false}]}]}In this case, files will be filtered such that:"subject" = "holy-bible" OR"person-of-interest" = "jesus christ" OR"genre" != "religion" OR"subject" = "tao-te-ching" AND "author" = "lao-tzu"Note that the top level of the query must be either an "OR" or "AND" array. Currently, nesting is limited to 3.
For tag blocks (those with "key", "value", and "negate" keys), the following typing rules apply:"key" isn't optional and must be astring"value" isn't optional and can beanyor list[any]"negate" is optional and must betrueorfalse. If present andtrue, then the filter block is negated in
the resulting query. It isfalseby default.When querying embeddings, you can optionally specify themedia_typeparameter in your request. By default (if
not set), it is equal to "TEXT". This means that the query will be performed over files that have
been parsed as text (for now, this covers all files except image files). If it is equal to "IMAGE",
the query will be performed over image files (for now,.jpgand.pngfiles). You can think of this
field as an additional filter on top of any filters set infile_idsandWhenhybrid_searchis set to true, a combination of keyword search and semantic search are used to rank
and select candidate embeddings during information retrieval. By default, these search methods are weighted
equally during the ranking process. To adjust the weight (or "importance") of each search method, you can use
thehybrid_search_tuning_parametersproperty. The description for the different tuning parameters are:weight_a: weight to assign to semantic searchweight_b: weight to assign to keyword searchYou must ensure thatsum(weight_a, weight_b,..., weight_n)for allnweights is equal to 1. The equality
has an error tolerance of 0.001 to account for possible floating point issues.In order to use hybrid search for a customer across a set of documents, two flags need to be enabled:Use the/modify_user_configurationendpoint to to enablesparse_vectorsfor the customer. The payload
body for this request is below:{
  "configuration_key_name": "sparse_vectors",
  "value": {
    "enabled": true
  }
}Make sure hybrid search is enabled for the documents across which you want to perform the search. For the/uploadfileendpoint, this can be done by setting the following query parameter:generate_sparse_vectors=trueCarbon supports multiple models for use in generating embeddings for files. For images, we support Vertex AI's
multimodal model; for text, we support OpenAI'stext-embedding-ada-002and Cohere's embed-multilingual-v3.0.
The model can be specified via theembedding_modelparameter (in the POST body for/embeddings, and a query
parameter in/uploadfile). If no model is supplied, thetext-embedding-ada-002is used by default. When performing
embedding queries, embeddings from files that used the specified model will be considered in the query.
For example, if files A and B have embeddings generated withOPENAI, and files C and D have embeddings generated withCOHERE_MULTILINGUAL_V3, then by default, queries will only consider files A and B. IfCOHERE_MULTILINGUAL_V3is
specified as theembedding_modelin/embeddings, then only files C and D will be considered. Make sure that
the set of all files you want considered for a query have embeddings generated via the same model. For now,do notsetVERTEX_MULTIMODALas anembedding_model. This model is used automatically by Carbon when it detects an image file.🛠️ Usageget_documents_response=carbon.embeddings.get_documents(query="a",k=1,tags={"key":"string_example",},query_vector=[3.14],file_ids=[1],parent_file_ids=[1],tags_v2={},include_tags=True,include_vectors=True,include_raw_file=True,hybrid_search=True,hybrid_search_tuning_parameters={"weight_a":0.5,"weight_b":0.5,},media_type="TEXT",embedding_model="OPENAI",)⚙️ Parametersquery:strQuery for which to get related chunks and embeddings.k:intNumber of related chunks to return.tags:GetEmbeddingDocumentsBodyTagsquery_vector:GetEmbeddingDocumentsBodyQueryVectorfile_ids:GetEmbeddingDocumentsBodyFileIdsparent_file_ids:GetEmbeddingDocumentsBodyParentFileIdstags_v2:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]A set of tags to limit the search to. Use this instead oftags, which is deprecated.include_tags:Optional[bool]Flag to control whether or not to include tags for each chunk in the response.include_vectors:Optional[bool]Flag to control whether or not to include embedding vectors in the response.include_raw_file:Optional[bool]Flag to control whether or not to include a signed URL to the raw file containing each chunk         in the response.hybrid_search:Optional[bool]Flag to control whether or not to perform hybrid search.hybrid_search_tuning_parameters:HybridSearchTuningParamsNullablemedia_type:FileContentTypesNullableembedding_model:EmbeddingGeneratorsNullable⚙️ Request BodyGetEmbeddingDocumentsBody🔄 ReturnDocumentResponseList🌐 Endpoint/embeddingspost🔙Back to Table of Contentscarbon.embeddings.get_embeddings_and_chunksRetrieve Embeddings And Content🛠️ Usageget_embeddings_and_chunks_response=carbon.embeddings.get_embeddings_and_chunks(filters={"user_file_id":1,"embedding_model":"OPENAI",},pagination={"limit":10,"offset":0,},order_by="created_at",order_dir="desc",include_vectors=False,)⚙️ Parametersfilters:EmbeddingsAndChunksFilterspagination:Paginationorder_by:EmbeddingsAndChunksOrderByColumnsorder_dir:OrderDirinclude_vectors:bool⚙️ Request BodyEmbeddingsAndChunksQueryInput🔄 ReturnEmbeddingsAndChunksResponse🌐 Endpoint/text_chunkspost🔙Back to Table of Contentscarbon.embeddings.upload_chunks_and_embeddingsUpload Chunks And Embeddings🛠️ Usageupload_chunks_and_embeddings_response=carbon.embeddings.upload_chunks_and_embeddings(embedding_model="OPENAI",chunks_and_embeddings=[{"file_id":1,"chunks_and_embeddings":[{"chunk_number":1,"chunk":"chunk_example","embedding":[3.14],}],}],overwrite_existing=False,)⚙️ Parametersembedding_model:EmbeddingGeneratorschunks_and_embeddings: List[SingleChunksAndEmbeddingsUploadInput]overwrite_existing:bool⚙️ Request BodyChunksAndEmbeddingsUploadInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/upload_chunks_and_embeddingspost🔙Back to Table of Contentscarbon.files.create_user_file_tagsA tag is a key-value pair that can be added to a file. This pair can then be used
for searches (e.g. embedding searches) in order to narrow down the scope of the search.
A file can have any number of tags. The following are reserved keys that cannot be used:db_embedding_idorganization_iduser_idorganization_user_file_idCarbon currently supports two data types for tag values -stringandlist<string>.
Keys can only bestring. If values other thanstringandlist<string>are used,
they're automatically converted to strings (e.g. 4 will become "4").🛠️ Usagecreate_user_file_tags_response=carbon.files.create_user_file_tags(tags={"key":"string_example",},organization_user_file_id=1,)⚙️ Parameterstags:OrganizationUserFileTagCreateTagsorganization_user_file_id:int⚙️ Request BodyOrganizationUserFileTagCreate🔄 ReturnUserFile🌐 Endpoint/create_user_file_tagspost🔙Back to Table of Contentscarbon.files.deleteDelete File Endpoint🛠️ Usagedelete_response=carbon.files.delete(file_id=1,)⚙️ Parametersfile_id:int🔄 ReturnGenericSuccessResponse🌐 Endpoint/deletefile/{file_id}delete🔙Back to Table of Contentscarbon.files.delete_file_tagsDelete File Tags🛠️ Usagedelete_file_tags_response=carbon.files.delete_file_tags(tags=["string_example"],organization_user_file_id=1,)⚙️ Parameterstags:OrganizationUserFileTagsRemoveTagsorganization_user_file_id:int⚙️ Request BodyOrganizationUserFileTagsRemove🔄 ReturnUserFile🌐 Endpoint/delete_user_file_tagspost🔙Back to Table of Contentscarbon.files.delete_manyDelete Files Endpoint🛠️ Usagedelete_many_response=carbon.files.delete_many(file_ids=[1],sync_statuses=["string_example"],delete_non_synced_only=False,)⚙️ Parametersfile_ids:DeleteFilesQueryInputFileIdssync_statuses: List[ExternalFileSyncStatuses]delete_non_synced_only:bool⚙️ Request BodyDeleteFilesQueryInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/delete_filespost🔙Back to Table of Contentscarbon.files.get_parsed_fileThis route is deprecated. Use/user_files_v2instead.🛠️ Usageget_parsed_file_response=carbon.files.get_parsed_file(file_id=1,)⚙️ Parametersfile_id:int🔄 ReturnPresignedURLResponse🌐 Endpoint/parsed_file/{file_id}get🔙Back to Table of Contentscarbon.files.get_raw_fileThis route is deprecated. Use/user_files_v2instead.🛠️ Usageget_raw_file_response=carbon.files.get_raw_file(file_id=1,)⚙️ Parametersfile_id:int🔄 ReturnPresignedURLResponse🌐 Endpoint/raw_file/{file_id}get🔙Back to Table of Contentscarbon.files.query_user_filesFor pre-filtering documents, usingtags_v2is preferred to usingtags(which is now deprecated). If bothtags_v2andtagsare specified,tagsis ignored.tags_v2enables
building complex filters through the use of "AND", "OR", and negation logic. Take the below input as an example:{"OR":[{"key":"subject","value":"holy-bible","negate":false},{"key":"person-of-interest","value":"jesus christ","negate":false},{"key":"genre","value":"religion","negate":true}{"AND":[{"key":"subject","value":"tao-te-ching","negate":false},{"key":"author","value":"lao-tzu","negate":false}]}]}In this case, files will be filtered such that:"subject" = "holy-bible" OR"person-of-interest" = "jesus christ" OR"genre" != "religion" OR"subject" = "tao-te-ching" AND "author" = "lao-tzu"Note that the top level of the query must be either an "OR" or "AND" array. Currently, nesting is limited to 3.
For tag blocks (those with "key", "value", and "negate" keys), the following typing rules apply:"key" isn't optional and must be astring"value" isn't optional and can beanyor list[any]"negate" is optional and must betrueorfalse. If present andtrue, then the filter block is negated in
the resulting query. It isfalseby default.🛠️ Usagequery_user_files_response=carbon.files.query_user_files(pagination={"limit":10,"offset":0,},order_by="created_at",order_dir="desc",filters={},include_raw_file=True,include_parsed_text_file=True,include_additional_files=True,)⚙️ Parameterspagination:Paginationorder_by:OrganizationUserFilesToSyncOrderByTypesorder_dir:OrderDirfilters:OrganizationUserFilesToSyncFiltersinclude_raw_file:Optional[bool]include_parsed_text_file:Optional[bool]include_additional_files:Optional[bool]⚙️ Request BodyOrganizationUserFilesToSyncQueryInput🔄 ReturnUserFilesV2🌐 Endpoint/user_files_v2post🔙Back to Table of Contentscarbon.files.query_user_files_deprecatedThis route is deprecated. Use/user_files_v2instead.🛠️ Usagequery_user_files_deprecated_response=carbon.files.query_user_files_deprecated(pagination={"limit":10,"offset":0,},order_by="created_at",order_dir="desc",filters={},include_raw_file=True,include_parsed_text_file=True,include_additional_files=True,)⚙️ Parameterspagination:Paginationorder_by:OrganizationUserFilesToSyncOrderByTypesorder_dir:OrderDirfilters:OrganizationUserFilesToSyncFiltersinclude_raw_file:Optional[bool]include_parsed_text_file:Optional[bool]include_additional_files:Optional[bool]⚙️ Request BodyOrganizationUserFilesToSyncQueryInput🔄 ReturnFilesQueryUserFilesDeprecatedResponse🌐 Endpoint/user_filespost🔙Back to Table of Contentscarbon.files.resyncResync File🛠️ Usageresync_response=carbon.files.resync(file_id=1,chunk_size=1,chunk_overlap=1,)⚙️ Parametersfile_id:intchunk_size:Optional[int]chunk_overlap:Optional[int]⚙️ Request BodyResyncFileQueryInput🔄 ReturnUserFile🌐 Endpoint/resync_filepost🔙Back to Table of Contentscarbon.files.uploadThis endpoint is used to directly upload local files to Carbon. ThePOSTrequest should be a multipart form request.
Note that theset_page_as_boundaryquery parameter is applicable only to PDFs for now. When this value is set,
PDF chunks are at most one page long. Additional information can be retrieved for each chunk, however, namely the coordinates
of the bounding box around the chunk (this can be used for things like text highlighting). Following is a description
of all possible query parameters:chunk_size: the chunk size (in tokens) applied when splitting the documentchunk_overlap: the chunk overlap (in tokens) applied when splitting the documentskip_embedding_generation: whether or not to skip the generation of chunks and embeddingsset_page_as_boundary: described aboveembedding_model: the model used to generate embeddings for the document chunksuse_ocr: whether or not to use OCR as a preprocessing step prior to generating chunks (only valid for PDFs currently)generate_sparse_vectors: whether or not to generate sparse vectors for the file. Required for hybrid search.prepend_filename_to_chunks: whether or not to prepend the filename to the chunk textCarbon supports multiple models for use in generating embeddings for files. For images, we support Vertex AI's
multimodal model; for text, we support OpenAI'stext-embedding-ada-002and Cohere's embed-multilingual-v3.0.
The model can be specified via theembedding_modelparameter (in the POST body for/embeddings, and a query
parameter in/uploadfile). If no model is supplied, thetext-embedding-ada-002is used by default. When performing
embedding queries, embeddings from files that used the specified model will be considered in the query.
For example, if files A and B have embeddings generated withOPENAI, and files C and D have embeddings generated withCOHERE_MULTILINGUAL_V3, then by default, queries will only consider files A and B. IfCOHERE_MULTILINGUAL_V3is
specified as theembedding_modelin/embeddings, then only files C and D will be considered. Make sure that
the set of all files you want considered for a query have embeddings generated via the same model. For now,do notsetVERTEX_MULTIMODALas anembedding_model. This model is used automatically by Carbon when it detects an image file.🛠️ Usageupload_response=carbon.files.upload(file=open("/path/to/file","rb"),chunk_size=1,chunk_overlap=1,skip_embedding_generation=False,set_page_as_boundary=False,embedding_model="OPENAI",use_ocr=False,generate_sparse_vectors=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersfile:IOchunk_size:Optional[int]Chunk size in tiktoken tokens to be used when processing file.chunk_overlap:Optional[int]Chunk overlap in tiktoken tokens to be used when processing file.skip_embedding_generation:boolFlag to control whether or not embeddings should be generated and stored             when processing file.set_page_as_boundary:boolFlag to control whether or not to set the a page's worth of content as the maximum             amount of content that can appear in a chunk. Only valid for PDFs. See description route description for             more information.embedding_model:TextEmbeddingGeneratorsEmbedding model that will be used to embed file chunks.use_ocr:boolWhether or not to use OCR when processing files. Only valid for PDFs. Useful for documents with             tables, images, and/or scanned text.generate_sparse_vectors:boolWhether or not to generate sparse vectors for the file. This isrequiredfor the file to be a             candidate for hybrid search.prepend_filename_to_chunks:boolWhether or not to prepend the file's name to chunks.max_items_per_chunk:Optional[int]Number of objects per chunk. For json files only.⚙️ Request BodyBodyCreateUploadFileUploadfilePost🔄 ReturnUserFile🌐 Endpoint/uploadfilepost🔙Back to Table of Contentscarbon.files.upload_from_urlCreate Upload File From Url🛠️ Usageupload_from_url_response=carbon.files.upload_from_url(url="string_example",file_name="string_example",chunk_size=1,chunk_overlap=1,skip_embedding_generation=False,set_page_as_boundary=False,embedding_model="OPENAI",generate_sparse_vectors=False,use_textract=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersurl:strfile_name:Optional[str]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:boolset_page_as_boundary:boolembedding_model:EmbeddingGeneratorsgenerate_sparse_vectors:booluse_textract:boolprepend_filename_to_chunks:boolmax_items_per_chunk:Optional[int]⚙️ Request BodyUploadFileFromUrlInput🔄 ReturnUserFile🌐 Endpoint/upload_file_from_urlpost🔙Back to Table of Contentscarbon.files.upload_textCarbon supports multiple models for use in generating embeddings for files. For images, we support Vertex AI's
multimodal model; for text, we support OpenAI'stext-embedding-ada-002and Cohere's embed-multilingual-v3.0.
The model can be specified via theembedding_modelparameter (in the POST body for/embeddings, and a query
parameter in/uploadfile). If no model is supplied, thetext-embedding-ada-002is used by default. When performing
embedding queries, embeddings from files that used the specified model will be considered in the query.
For example, if files A and B have embeddings generated withOPENAI, and files C and D have embeddings generated withCOHERE_MULTILINGUAL_V3, then by default, queries will only consider files A and B. IfCOHERE_MULTILINGUAL_V3is
specified as theembedding_modelin/embeddings, then only files C and D will be considered. Make sure that
the set of all files you want considered for a query have embeddings generated via the same model. For now,do notsetVERTEX_MULTIMODALas anembedding_model. This model is used automatically by Carbon when it detects an image file.🛠️ Usageupload_text_response=carbon.files.upload_text(contents="string_example",name="string_example",chunk_size=1,chunk_overlap=1,skip_embedding_generation=False,overwrite_file_id=1,embedding_model="OPENAI",generate_sparse_vectors=False,)⚙️ Parameterscontents:strname:Optional[str]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:booloverwrite_file_id:Optional[int]embedding_model:EmbeddingGeneratorsNullablegenerate_sparse_vectors:Optional[bool]⚙️ Request BodyRawTextInput🔄 ReturnUserFile🌐 Endpoint/upload_textpost🔙Back to Table of Contentscarbon.health.checkHealth🛠️ Usagecheck_response=carbon.health.check()🌐 Endpoint/healthget🔙Back to Table of Contentscarbon.integrations.connect_freshdeskRefer this article to obtain an API keyhttps://support.freshdesk.com/en/support/solutions/articles/215517.
Make sure that your API key has the permission to read solutions from your account and you are on apaidplan.
Once you have an API key, you can make a request to this endpoint along with your freshdesk domain. This will
trigger an automatic sync of the articles in your "solutions" tab. Additional parameters below can be used to associate
data with the synced articles or modify the sync behavior.🛠️ Usageconnect_freshdesk_response=carbon.integrations.connect_freshdesk(domain="string_example",api_key="string_example",tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,)⚙️ Parametersdomain:strapi_key:strtags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsNullablegenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]⚙️ Request BodyFreshDeskConnectRequest🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/freshdeskpost🔙Back to Table of Contentscarbon.integrations.create_aws_iam_userCreate a new IAM user with permissions to:List all buckets.Read from the specific buckets and objects to sync with Carbon. Ensure any future buckets or objects carry 
the same permissions.Once created, generate an access key for this user and share the credentials with us. We recommend testing this key beforehand.🛠️ Usagecreate_aws_iam_user_response=carbon.integrations.create_aws_iam_user(access_key="string_example",access_key_secret="string_example",)⚙️ Parametersaccess_key:straccess_key_secret:str⚙️ Request BodyS3AuthRequest🔄 ReturnOrganizationUserDataSourceAPI🌐 Endpoint/integrations/s3post🔙Back to Table of Contentscarbon.integrations.get_oauth_urlGet Oauth Url🛠️ Usageget_oauth_url_response=carbon.integrations.get_oauth_url(service="GOOGLE_DRIVE",tags=None,scope="string_example",chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",zendesk_subdomain="string_example",microsoft_tenant="string_example",sharepoint_site_name="string_example",confluence_subdomain="string_example",generate_sparse_vectors=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersservice:DataSourceTypetags:Union[bool, date, datetime, dict, float, int, list, str, None]scope:Optional[str]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsNullablezendesk_subdomain:Optional[str]microsoft_tenant:Optional[str]sharepoint_site_name:Optional[str]confluence_subdomain:Optional[str]generate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]max_items_per_chunk:Optional[int]⚙️ Request BodyOAuthURLRequest🌐 Endpoint/integrations/oauth_urlpost🔙Back to Table of Contentscarbon.integrations.list_confluence_pagesTo begin listing a user's Confluence pages, at least adata_source_idof a connected
Confluence account must be specified. This base request returns a list of root pages for
every space the user has access to in a Confluence instance. To traverse further down
the user's page directory, additional requests to this endpoint can be made with the samedata_source_idand withparent_idset to the id of page from a previous request. For
convenience, thehas_childrenproperty in each directory item in the response list will
flag which pages will return non-empty lists of pages when set as theparent_id.🛠️ Usagelist_confluence_pages_response=carbon.integrations.list_confluence_pages(data_source_id=1,parent_id="string_example",)⚙️ Parametersdata_source_id:intparent_id:Optional[str]⚙️ Request BodyListRequest🔄 ReturnListResponse🌐 Endpoint/integrations/confluence/listpost🔙Back to Table of Contentscarbon.integrations.list_data_source_itemsList Data Source Items🛠️ Usagelist_data_source_items_response=carbon.integrations.list_data_source_items(data_source_id=1,parent_id="string_example",pagination={"limit":10,"offset":0,},)⚙️ Parametersdata_source_id:intparent_id:Optional[str]pagination:Pagination⚙️ Request BodyListDataSourceItemsRequest🔄 ReturnListDataSourceItemsResponse🌐 Endpoint/integrations/items/listpost🔙Back to Table of Contentscarbon.integrations.list_foldersAfter connecting your Outlook account, you can use this endpoint to list all of your folders on outlook. This includes
both system folders like "inbox" and user created folders.🛠️ Usagelist_folders_response=carbon.integrations.list_folders()🌐 Endpoint/integrations/outlook/user_foldersget🔙Back to Table of Contentscarbon.integrations.list_labelsAfter connecting your Gmail account, you can use this endpoint to list all of your labels. User created labels
will have the type "user" and Gmail's default labels will have the type "system"🛠️ Usagelist_labels_response=carbon.integrations.list_labels()🌐 Endpoint/integrations/gmail/user_labelsget🔙Back to Table of Contentscarbon.integrations.sync_confluenceAfter listing pages in a user's Confluence account, the set of selected pageidsand the
connected account'sdata_source_idcan be passed into this endpoint to sync them into
Carbon. Additional parameters listed below can be used to associate data to the selected
pages or alter the behavior of the sync.🛠️ Usagesync_confluence_response=carbon.integrations.sync_confluence(data_source_id=1,ids=["string_example"],tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersdata_source_id:intids:SyncFilesRequestIdstags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsNullablegenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]max_items_per_chunk:Optional[int]⚙️ Request BodySyncFilesRequest🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/confluence/syncpost🔙Back to Table of Contentscarbon.integrations.sync_data_source_itemsSync Data Source Items🛠️ Usagesync_data_source_items_response=carbon.integrations.sync_data_source_items(data_source_id=1,)⚙️ Parametersdata_source_id:int⚙️ Request BodySyncDirectoryRequest🔄 ReturnOrganizationUserDataSourceAPI🌐 Endpoint/integrations/items/syncpost🔙Back to Table of Contentscarbon.integrations.sync_filesSync Files🛠️ Usagesync_files_response=carbon.integrations.sync_files(data_source_id=1,ids=["string_example"],tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersdata_source_id:intids:SyncFilesRequestIdstags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsNullablegenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]max_items_per_chunk:Optional[int]⚙️ Request BodySyncFilesRequest🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/files/syncpost🔙Back to Table of Contentscarbon.integrations.sync_gmailOnce you have successfully connected your gmail account, you can choose which emails to sync with us
using the filters parameter. Filters is a JSON object with key value pairs. It also supports AND and OR operations.
For now, we support a limited set of keys listed below.label: Inbuilt Gmail labels, for example "Important" or a custom label you created.afterorbefore: A date in YYYY/mm/dd format (example 2023/12/31). Gets emails after/before a certain date.
You can also use them in combination to get emails from a certain period.is: Can have the following values - starred, important, snoozed, and unreadUsing keys or values outside of the specified values can lead to unexpected behaviour.An example of a basic query with filters can be{"filters":{"key":"label","value":"Test"}}Which will list all emails that have the label "Test".You can use AND and OR operation in the following way:{"filters":{"AND":[{"key":"after","value":"2024/01/07"},{"OR":[{"key":"label","value":"Personal"},{"key":"is","value":"starred"}]}]}}This will return emails after 7th of Jan that are either starred or have the label "Personal".
Note that this is the highest level of nesting we support, i.e. you can't add more AND/OR filters within the OR filter
in the above example.🛠️ Usagesync_gmail_response=carbon.integrations.sync_gmail(filters={},tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,)⚙️ Parametersfilters:Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]tags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsgenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]⚙️ Request BodyGmailSyncInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/gmail/syncpost🔙Back to Table of Contentscarbon.integrations.sync_outlookOnce you have successfully connected your Outlook account, you can choose which emails to sync with us
using the filters and folder parameter. "folder" should be the folder you want to sync from Outlook. By default
we get messages from your inbox folder.Filters is a JSON object with key value pairs. It also supports AND and OR operations.
For now, we support a limited set of keys listed below.category: Custom categories that you created in Outlook.afterorbefore: A date in YYYY/mm/dd format (example 2023/12/31). Gets emails after/before a certain date. You can also use them in combination to get emails from a certain period.is: Can have the following values: flaggedAn example of a basic query with filters can be{"filters":{"key":"category","value":"Test"}}Which will list all emails that have the category "Test".Specifying a custom folder in the same query{"folder":"Folder Name","filters":{"key":"category","value":"Test"}}You can use AND and OR operation in the following way:{"filters":{"AND":[{"key":"after","value":"2024/01/07"},{"OR":[{"key":"category","value":"Personal"},{"key":"category","value":"Test"},]}]}}This will return emails after 7th of Jan that have either Personal or Test as category.
Note that this is the highest level of nesting we support, i.e. you can't add more AND/OR filters within the OR filter
in the above example.🛠️ Usagesync_outlook_response=carbon.integrations.sync_outlook(filters={},tags={},folder="Inbox",chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,)⚙️ Parametersfilters:Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]tags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]folder:Optional[str]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsgenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]⚙️ Request BodyOutlookSyncInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/outlook/syncpost🔙Back to Table of Contentscarbon.integrations.sync_rss_feedRss Feed🛠️ Usagesync_rss_feed_response=carbon.integrations.sync_rss_feed(url="string_example",tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,)⚙️ Parametersurl:strtags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsgenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]⚙️ Request BodyRSSFeedInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/rss_feedpost🔙Back to Table of Contentscarbon.integrations.sync_s3_filesAfter optionally loading the items via /integrations/items/sync and integrations/items/list, use the bucket name
and object key as the ID in this endpoint to sync them into Carbon. Additional parameters below can associate
data with the selected items or modify the sync behavior🛠️ Usagesync_s3_files_response=carbon.integrations.sync_s3_files(ids=[{}],tags={},chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,embedding_model="OPENAI",generate_sparse_vectors=False,prepend_filename_to_chunks=False,max_items_per_chunk=1,)⚙️ Parametersids: List[S3GetFileInput]tags:Optional[Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]embedding_model:EmbeddingGeneratorsgenerate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]max_items_per_chunk:Optional[int]⚙️ Request BodyS3FileSyncInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/integrations/s3/filespost🔙Back to Table of Contentscarbon.organizations.getGet Organization🛠️ Usageget_response=carbon.organizations.get()🔄 ReturnOrganizationResponse🌐 Endpoint/organizationget🔙Back to Table of Contentscarbon.users.getUser Endpoint🛠️ Usageget_response=carbon.users.get(customer_id="string_example",)⚙️ Parameterscustomer_id:str⚙️ Request BodyUserRequestContent🔄 ReturnUserResponse🌐 Endpoint/userpost🔙Back to Table of Contentscarbon.users.toggle_user_featuresToggle User Features🛠️ Usagetoggle_user_features_response=carbon.users.toggle_user_features(configuration_key_name="string_example",value={},)⚙️ Parametersconfiguration_key_name:strvalue:Dict[str, Union[bool, date, datetime, dict, float, int, list, str, None]]⚙️ Request BodyModifyUserConfigurationInput🔄 ReturnGenericSuccessResponse🌐 Endpoint/modify_user_configurationpost🔙Back to Table of Contentscarbon.utilities.fetch_urlsExtracts all URLs from a webpage.Args:
url (str): URL of the webpageReturns:
FetchURLsResponse: A response object with a list of URLs extracted from the webpage and the webpage content.🛠️ Usagefetch_urls_response=carbon.utilities.fetch_urls(url="url_example",)⚙️ Parametersurl:str🔄 ReturnFetchURLsResponse🌐 Endpoint/fetch_urlsget🔙Back to Table of Contentscarbon.utilities.fetch_youtube_transcriptsFetches english transcripts from YouTube videos.Args:
id (str): The ID of the YouTube video.
raw (bool): Whether to return the raw transcript or not. Defaults to False.Returns:
dict: A dictionary with the transcript of the YouTube video.🛠️ Usagefetch_youtube_transcripts_response=carbon.utilities.fetch_youtube_transcripts(id="id_example",raw=False,)⚙️ Parametersid:strraw:bool🔄 ReturnYoutubeTranscriptResponse🌐 Endpoint/fetch_youtube_transcriptget🔙Back to Table of Contentscarbon.utilities.process_sitemapRetrieves all URLs from a sitemap, which can subsequently be utilized with ourweb_scrapeendpoint.🛠️ Usageprocess_sitemap_response=carbon.utilities.process_sitemap(url="url_example",)⚙️ Parametersurl:str🌐 Endpoint/process_sitemapget🔙Back to Table of Contentscarbon.utilities.scrape_sitemapExtracts all URLs from a sitemap and performs a web scrape on each of them.Args:
sitemap_url (str): URL of the sitemapReturns:
dict: A response object with the status of the scraping job message.-->🛠️ Usagescrape_sitemap_response=carbon.utilities.scrape_sitemap(url="string_example",tags={"key":"string_example",},max_pages_to_scrape=100,chunk_size=1500,chunk_overlap=20,skip_embedding_generation=False,enable_auto_sync=False,generate_sparse_vectors=False,prepend_filename_to_chunks=False,html_tags_to_skip=[],css_classes_to_skip=[],css_selectors_to_skip=[],)⚙️ Parametersurl:strtags:SitemapScrapeRequestTagsmax_pages_to_scrape:Optional[int]chunk_size:Optional[int]chunk_overlap:Optional[int]skip_embedding_generation:Optional[bool]enable_auto_sync:Optional[bool]generate_sparse_vectors:Optional[bool]prepend_filename_to_chunks:Optional[bool]html_tags_to_skip:SitemapScrapeRequestHtmlTagsToSkipcss_classes_to_skip:SitemapScrapeRequestCssClassesToSkipcss_selectors_to_skip:SitemapScrapeRequestCssSelectorsToSkip⚙️ Request BodySitemapScrapeRequest🌐 Endpoint/scrape_sitemappost🔙Back to Table of Contentscarbon.utilities.scrape_webConduct a web scrape on a given webpage URL. Our web scraper is fully compatible with JavaScript and supports recursion depth, enabling you to efficiently extract all content from the target website.🛠️ Usagescrape_web_response=carbon.utilities.scrape_web(body=[{"url":"url_example","recursion_depth":3,"max_pages_to_scrape":100,"chunk_size":1500,"chunk_overlap":20,"skip_embedding_generation":False,"enable_auto_sync":False,"generate_sparse_vectors":False,"prepend_filename_to_chunks":False,"html_tags_to_skip":[],"css_classes_to_skip":[],"css_selectors_to_skip":[],}],)⚙️ Request BodyUtilitiesScrapeWebRequest🌐 Endpoint/web_scrapepost🔙Back to Table of Contentscarbon.utilities.search_urlsPerform a web search and obtain a list of relevant URLs.As an illustration, when you perform a search for “content related to MRNA,” you will receive a list of links such as the following:- https://tomrenz.substack.com/p/mrna-and-why-it-matters

- https://www.statnews.com/2020/11/10/the-story-of-mrna-how-a-once-dismissed-idea-became-a-leading-technology-in-the-covid-vaccine-race/

- https://www.statnews.com/2022/11/16/covid-19-vaccines-were-a-success-but-mrna-still-has-a-delivery-problem/

- https://joomi.substack.com/p/were-still-being-misled-about-howSubsequently, you can submit these links to the web_scrape endpoint in order to retrieve the content of the respective web pages.Args:
query (str): Query to search forReturns:
FetchURLsResponse: A response object with a list of URLs for a given search query.🛠️ Usagesearch_urls_response=carbon.utilities.search_urls(query="query_example",)⚙️ Parametersquery:str🔄 ReturnFetchURLsResponse🌐 Endpoint/search_urlsget🔙Back to Table of Contentscarbon.webhooks.add_urlAdd Webhook Url🛠️ Usageadd_url_response=carbon.webhooks.add_url(url="string_example",)⚙️ Parametersurl:str⚙️ Request BodyAddWebhookProps🔄 ReturnWebhook🌐 Endpoint/add_webhookpost🔙Back to Table of Contentscarbon.webhooks.delete_urlDelete Webhook Url🛠️ Usagedelete_url_response=carbon.webhooks.delete_url(webhook_id=1,)⚙️ Parameterswebhook_id:int🔄 ReturnGenericSuccessResponse🌐 Endpoint/delete_webhook/{webhook_id}delete🔙Back to Table of Contentscarbon.webhooks.urlsWebhook Urls🛠️ Usageurls_response=carbon.webhooks.urls(pagination={"limit":10,"offset":0,},order_by="created_at",order_dir="desc",filters={"ids":[],},)⚙️ Parameterspagination:Paginationorder_by:WebhookOrderByColumnsorder_dir:OrderDirfilters:WebhookFilters⚙️ Request BodyWebhookQueryInput🔄 ReturnWebhookQueryResponse🌐 Endpoint/webhookspost🔙Back to Table of ContentsAuthorThis Python package is automatically generated byKonfig