Package: tgmount
OverviewCreates virtual file system with files posted on telegram.VERY ALPHA SO FARTable of ContentsInstallationRequirementsBasic usageMounting multiple entitiesSample configClient commandsmountmount configlist dialogslist documentsdownloadConfig file structurePlaying flac and mp3 from a zip archiveKnown bugsRequirementsLinuxPython 3.10 (not sure about 3.9 and less)Installation:pip install tgmountBasic usageTo obtain your API id followofficial manual.  Running the program for the first time will require authentication.$ export TGAPP=1234567:deadbeef0d04a3efe93e1af778773d6f0 TGSESSION=tgfsTo mount a channel/chat/grouptgmount mount tgmounttestingchannel ~/mnt/tgmount1/To mount an entity that doesn't have a username you will need to get its id.tgmountlistdialogs|grep'my friends private chat'To mount zip files as directories useUnpackedZipproducertgmount mount tgmounttestingchannel ~/mnt/tgmount1/ --producer UnpackedZipUse config file to create a more complex vfs structuretgmount mount tgmounttestingchannel ~/mnt/tgmount1/ --root-config examples/root_config.yamlMounting multiple entitiesTo mount multiple entities usemount-configcommandtgmount mount-config examples/config.yamlSample config# can be overwritten by --mount-dir argumentmount_dir:/home/horn/mnt/tgmount1client:session:tgfsapi_id:123api_hash:deadbeed121212121#message_sources:ru2chmu:entity:ru2chmuupdates:Falselimit:1000friends:entity:-388004022limit:1000caches:memory1:type:memorycapacity:300MBblock_size:128KBroot:muzach:# A document will not be mounted more than once when it appears in a# different messages. `recursive` means this filter will also be applied# down the folders treefilter:{filter:OnlyUniqueDocs,recursive:True}# Messages from `ru2chmu` will be used to produce content in the nested folderssource:{source:ru2chmu,recursive:True}# creates subfolder named `music`music:filter:# the directory will contain music and zip archivesUnion:[MessageWithMusic,MessageWithZip]# zip archives will be mounted as foldersproducer:UnpackedZip# using cache speeds up reading from the archivescache:memory1texts:# messages with textfilter:MessageWithText# this commands tgmount to treat messages with both document and text# as text filestreat_as:MessageWithTextfriends:source:{source:friends,recursive:True}music-by-senders:producer:# this producer creates a separate directory for every sender in the entityBySender:dir_structure:# these directories will only contain musicfilter:MessageWithMusicliked-music:# this directory will be containing all music with thumb up reactionfilter:And:-MessageWithMusic-ByReaction:reaction:👍images:filter:Union:[MessageWithCompressedPhoto,MessageWithDocumentImage]More about config structure read inConfig file structureClient commandstgmount mounttgmount mount [--filter FILTER] [--root-config ROOT_CONFIG]
[--producer PRODUCER] [--offset-date OFFSET_DATE] [--offset-id OFFSET_ID] 
[--max-id MAX_ID] [--min-id MIN_ID] [--wait_time WAIT_TIME] [--limit LIMIT] 
[--reply-to REPLY_TO] [--from-user FROM_USER] [--reverse] [--mount-texts] [--no-updates] 
[--debug-fuse] [--min-tasks MIN_TASKS] entity mount-dirDefine the structure of the mounted folder by one of these options--producer PRODUCER
--root-config ROOT_CONFIGAvailable producers:PlainDir# just a list of files (default)UnpackedZip# PlainDir but zips are mounted as foldersBySender# files grouped in folders by senderByForward# forwarded files grouped by source entityByPerformer# music grouped by performersByReactions# files grouped by reactionThe following arguments work as described inTelegramClient.get_messages.--filter [FILTER]
--offset-date OFFSET_DATE
--offset-id OFFSET_ID
--max-id MAX_ID
--min-id MIN_ID
--wait_time WAIT_TIME
--limit LIMIT
--reply-to REPLY_TO
--from-user FROM_USER
--reverseAvailabletelegram filters:InputMessagesFilterDocumentInputMessagesFilterPhotosInputMessagesFilterVideoInputMessagesFilterPhotoVideoInputMessagesFilterUrlInputMessagesFilterGifInputMessagesFilterVoiceInputMessagesFilterMusicInputMessagesFilterRoundVoiceInputMessagesFilterRoundVideoInputMessagesFilterMyMentionsUsing these filter speeds up fetching process but these filter cannot be composed.If you don't need updates--no-updatesIf you want to also to mount text messages as text files--mount-textsOther arguments--debug-fuse
--min-tasks MIN_TASKStgmount mount-configtgmount mount-config [--mount-dir MOUNT_DIR] CONFIG_FILE MOUNT_DIRtgmount list dialogstgmount list dialogstgmount list documentstgmount list documents [--filter FILTER] [--offset-date OFFSET_DATE] [--offset-id OFFSET_ID]
[--max-id MAX_ID] [--min-id MIN_ID] [--wait_time WAIT_TIME] [--limit LIMIT] 
[--reply-to REPLY_TO] [--from-user FROM_USER] [--reverse] [--json]
[--print-message] [--include-unsupported] [--only-unsupported] [--all-types]
[--only-unique-docs] entity--print-messageInclude stringified message object in the output--all-typesPrint all classes a message matches--only-unique-docsExclude repeating documents--include-unsupportedInclude messages that are not supported for mounting--only-unsupportedPrint only them--jsonPrint in json formattgmount downloadtgmount download [--output-dir OUTPUT_DIR] [--keep-filename] [--request-size REQUEST_SIZE] entity ids [ids ...]--keep-filenameKeep original filenames--output-dirDestination folder for files--request_sizeHow much data to fetch per requestentityEntity to download fromidsMessages idsExample:tgmount download -O /tmp -R 256KB tgmounttestingchannel 532 11 51 18Im combination withlist documentstgmountdownloadru_python$(tgmountlistdocumentsru_python--filterInputMessagesFilterDocument--limit10--json|jq'.[]|.id')-O/tmpConfig file structureConfig file has the following sections:clientmessage_sourcescachesrootcachessection is optional.Top level properties# optional. can be overwritten by --mount-dir argumentmount_dir:~/mnt/tgmountclientContains settings for the telegram clientclient:# telethon session namesession:session_name# telegram api credentialsapi_id:intapi_hash:str# optional fieldrequest_size:128KB# optional field. Default: Falseuse_ipv6:Truemessage_sourcesA message source defines a list of messages that will be used in vfs tree construction. Every message source is a separateTelegramClient.get_messagesrequest. Message source is also subscribed to events of posting, removing and editing messages in the entity it is sourced from.message_sources:# key defines id of the message source to reference in the `root` sectionsource1:# channel/group/chat id to fetch messages from# string or intentity:tgmounttestingchannel# all the following fields are optional# whether to listen for updates. Default: trueupdates:True# Filter for message types. If not set all the messages types including text# messages will be fetchedfilter:MessageWithMusic# limits the number of messageslimit:1000# format is `31/12/2023` or '31/12/2023 13:00'offset_date:`31/12/2023`offset_id:0min_id:0max_id:0wait_time:Nonereply_to:intfrom_user:str | intreverse:FalsecachesDefines cache storages for documents. Cached parts of a document will not be fetched twice. Usually this is not needed because OS file system does caching by itself. Cache is needed in couple withUnpackedZipproducer since the OS file system cache is not applied in case of using this producer.caches:# the key defines cache id to be referenced in `root` sectioncache1:# currently only memory cache is supportedtype:memory# The size of the cachecapacity:300MB# optional block size, default: 128KBblock_size:256KBrootThis section defines the structure of the mounted folder.root:# optional. sets the message source for the current directory. If this is not# set and there is no recursive filter has been defined before, the folder# will not contain any filessource:source1source:{source:source1}# sets the message source for the current and for nested folderssource:{source:source1,recursive:True}# optional. sets a filter for the current folder. Default is no filterfilter:MessageWithMusicfilter:{filter:MessageWithMusic}# sets a filter for the current folder and subfoldersfilter:{filter:MessageWithMusic,recursive:True}# sets a filter for the current folder and subfolders overwriting another# recursive filter if anyfilter:{filter:MessageWithMusic,overwright:True,recursive:True}# the following combines multiple filters. Only messages that match every filter# in the list will pass. The filter below allows all documents that# that are not video, photo or audio and not a zip filefilter:-MessageWithOtherDocument-Not:-ByExtension:.zip# in one linefilter:{filter:[MessageWithOtherDocument,Not:{ByExtension:.zip}],overwright:True,recursive:True}# defines a producer that controls the content of the folder.# Default is PlainDirproducer:BySender# producer may have propertiesproducer:BySender:dir_structure:music:filter:MessageWithMusicvoices:filter:MessageWithVoiceuse_get_sender:true# sets a cache for the current folder# referencing a cache defined in `caches` foldercache:memory1# dynamically creates a cache to use in this foldercache:type:memorycapacity:300MB# optional. wrapper that modifies the resulting content of the folderwrapper:ExcludeEmptyDirs# optional. Defines the priority of how to classify a message if multiple classes# match its type. E.g. a message with both a document and a text messagetreat_as:MessageWithText# to define subfoldersdocuments:# 'documents' folder will only contain the two following subfoldersdocs_from_source1:source:source1filter:MessageWithDocumentdocs_from_source2:source:source2filter:MessageWithDocumentsourceMessage source is a list of messages which is used to produce a directory content. Message source is initialized from get_messages() request and is updated by events of posting message, removing message and editing message in the corresponding entity.Producer is subscribed to a message source and takes care of the directory it is responsible for. It manages it by adding and removing files and subfolders.The content of a folder is defined by a combination of propertiessource,filter,producerandtreat_as.This will create a tree of empty foldersroot:everything:photos:texts:round-and-voice:rounds:voices:The config will result into/everything
/photos
/texts
/round-and-voice
/round-and-voice/rounds
/round-and-voice/voicesTo fill the directories with files we need to specify a source for every folder that is supposed to contain filesroot:everything:source:source1photos:source:source1texts:source:source1round-and-voice:rounds:source:source1voices:source:source1In result every directory that hassourceproperty will contain all the files from the specified source.Let's add filtersroot:everything:# don't need filter heresource:source1photos:source:source1filter:MessageWithCompressedPhototexts:source:source1filter:MessageWithTexttreats_as:MessageWithTextround-and-voice:rounds:source:source1filter:MessageWithKruzhochekvoices:source:source1filter:MessageWithVoiceAs soon as the only source used in the structure is "source1" we can get rid of repeating it by usingrecursiveproperty ofsource.root:source:{source:source1,recursive:True}everything:filter:Allphotos:filter:MessageWithCompressedPhototexts:filter:MessageWithTexttreats_as:MessageWithTextround-and-voice:rounds:filter:MessageWithKruzhochekvoices:filter:MessageWithVoiceNote thatThe root itself will not contain any files because source withrecursiveflag doesn't trigger file producingWe had to specifyfilterin "everything" to trigger file producer. For the same effect we could have specified a producer instead.everything:# triggers producing from the recursive sourceproducer:PlainDirThe complete rules:A folder will be produced with content from a message source in cases when:source is specified and it's not recursiverecursive source is in the context andfilterproperty specified and it's not recursiverecursive source is in the context andproducerprop is specifiedfilterBy message type:MessageWithDocument# Message with a document attached (message with compressed#  image doesn't match)MessageWithCompressedPhoto# with a compressed image (photo)MessageDownloadable# `MessageWithDocument` or `MessageWithCompressedPhoto`MessageWithAnimated# stickers, gifsMessageWithAudio# voices and musicMessageWithVoice# voiceMessageWithKruzhochek# round videoMessageWithDocumentImage# uncompressed imageMessageWithFilename# document with a filename attributeMessageWithMusic# musicMessageWithVideo# round video, video documents, stickers, gifsMessageWithVideoFile# video documentsMessageWithSticker# stickerMessageWithOtherDocument# Any document that doesn't fall in the previous categoriesMessageWithZip# zip fileMessageWithText# message with text messageMessageWithoutDocument# message with no document and no photoMessageWithReactions# message with reactionsMessageForwarded# forwarded message# Telegram filtersInputMessagesFilterPhotos# MessageWithCompressedPhotoInputMessagesFilterVideo# MessageWithVideoInputMessagesFilterPhotoVideo# MessageWithCompressedPhoto | MessageWithVideoInputMessagesFilterDocument# MessageWithOtherDocument | MessageWithDocumentImageInputMessagesFilterGif# MessageWithAnimatedInputMessagesFilterVoice# MessageWithVoiceInputMessagesFilterMusic# MessageWithMusicInputMessagesFilterRoundVideo# MessageWithKruzhochekInputMessagesFilterRoundVoice# MessageWithKruzhochek | MessageWithVoiceOther filters# Filter wrapper to reverse a filter.Not:MessageWithReactions# Combines multiple filters. If any matchesUnion:-MessageWithDocumentImage-MessageWithCompressedPhoto# Combines multiple filters. If every matchesAnd:-MessageForwarded-MessageWithVideo# same asfilter:[MessageForwarded,MessageWithVideo]# takes first `count` messagesFirst:count:10# takes last `count` messagesLast:count:10# Filter by a filename extensionByExtension:.zip# will only leave unique docsOnlyUniqueDocs:# optional. Control which document, first appeared or last appeared, will stay.# default: firstpicker:lastpicker:first# passthrough filter. Used to trigger tgmount to produce content in the folder# or to reset recursive filterAll# sequentially filters messages. E.g. last 10 unique documentsSeq:-MessageWithDocument-OnlyUniqueDocs-Last:10# matches reactionsByReaction:reaction:👍# optional. default: 1minimum:5producerPlainDirBySenderByForwardByPerformerByReactionsSysInfoUnpackedZipPlaying flac and mp3 from a zip archiveSeeking in files which are stored in a zip archive only works by reading the
offset bytes.id3v1 tags are stored in the end of a media file :)https://github.com/quodlibet/mutagen/blob/master/mutagen/id3/_id3v1.py#L34And most of the players try to read it. So just adding a mp3 or flac
to a player will fetch the whole file from the telegram cloud.In current moment this is solved by custom read function for mp3 and flac files
in archives. Thereadcall returns 4096 zero bytes whenless thanmax_total_read = 128KBbytes has been read from the file so farfile_size - offset < distance_to_file_end = 16KBsize == 4096(usually players read this amount looking for id3v1 (requires
further investigation to find a less hacky way))SeeFileContentZipFixingId3v1classTo disable this behavior use--no-fix-id3v1argument withmountcommand.
In case of mounting a config setfix_id3v1property ofUnpackedZipto False:producer:{UnpackedZip:{fix_id3v1:False}}Known bugsNo updates received during reconnectionCombination of--filter,--offset-dateand--reversealways returns empty result