Package: signalation
signal-kafka-producer a.k.a. signalationMotivation|Installation|Implementation DetailsPython package to produce messages from yoursignalaccount tokafkaby querying thedockerized signal messenger API.Motivation - Why should I use it?After starting thesignal-kafka-producerall your signal messages (sent and received) are produced onto a kafka topic.
As a result, there are two main advantages:Messages do not get lost, seeKeep (Ephemeral) Messages in Message QueueUseKafka Connectorsor consumers for your use case, seeKafka Consumers/ConnectorsKeep (Ephemeral) Messages in Message QueueWe've all been there: You are searching for some information and cannot find the corresponding message.
Maybe you've switched your phone, phone number or some of your contacts simply loveSignal's ephemeral/self-destructing messages- in any casesignal-kafka-producercomes to the rescue. Any message is stored
in a kafka topic and YOU are the maintainer of it.Note: Messages will be deleted in Kafka as well, seeRetention Policy.
However, you can either set infinite time or use a Kafka DB Sink Connector to store your messages in a database.Kafka Consumers/ConnectorsHaving your signal message in a Kafka Topic comes with all kafka associated benefits:Real-time processing:
Depending on your use-case, you can write consumers that can act on the messages in real-time, e.g.,A service that sends answers from ChatGPT whenever a message starts withChatGPT please help:A service that forwards messages toNote to Selfwhenever a self-destructing message is received.Flexibility:
Kafka Topics can be integrated using other tools such asKafka Connectors, e.g.,
you could use a Kafka DB Sink Connector to store your messages in a database.Installation - How can I use it?For running thesignal-kafka-producer, you'll need to have access to a running instance ofkafkaandsignal. If you do not have that go toComplete Installation including Dockerized Services,
otherwise you can directly usePip Installation.Pip InstallationpipinstallsignalationThe producer can then be executed viasignal-kafka-producer--env_file_path.envwhere the.envfile should have the following contentATTACHMENT_FOLDER_PATH=<folderpathinwhichattachmentsshallbestored># Signal configurationSIGNAL__REGISTERED_NUMBER=<yourphonenumber>SIGNAL__IP_ADRESS=<ipaddressofsignalrestapi>SIGNAL__PORT=<portofsignalrestapi>SIGNAL__TIMEOUT_IN_S=<signalrequesttimeoutinseconds>SIGNAL__RECEIVE_IN_S=<signalrequestintervalinseconds># Kafka configurationKAFKA__SERVER__PORT=<kafkabootstrapserverport>Complete Installation including Dockerized ServicesClone Repository, Install Python Package and Create Configurationgitclonegit@github.com:borea17/signal-kafka-producer.gitcdsignal-kafka-producer
pipinstall.In order to run the dockerized services (signal messenger and kafka) as well as the producer service,
you need to create a.envfile with the following contentATTACHMENT_FOLDER_PATH=./attachments# Signal configurationSIGNAL__REGISTERED_NUMBER=+49000000SIGNAL__IP_ADRESS=127.0.0.1SIGNAL__PORT=8080SIGNAL__TIMEOUT_IN_S=90SIGNAL__RECEIVE_IN_S=1# Kafka configurationKAFKA__UI__PORT=8081KAFKA__SERVER__PORT=9092KAFKA__ZOOKEEPER__PORT=2181Note: You'll need to replaceSIGNAL__REGISTERED_NUMBERwith your phone number. Of course, you are free to adjust
ports and timeouts / waiting times to your needs.Run Dockerized ServicesIn order to run thedockerized signal messengerand a dockerized kafka
(using your previously defined variables), you simply need to rundocker-compose-ftests/dockerized_services/signal/docker-compose.yml--env-file.envup-d
docker-compose-ftests/dockerized_services/kafka/docker-compose.yml--env-file.envup-dNote: Adjust paths accordingly.Start Producer via CLIsignal-kafka-producer--env_file_path.envNote: You'll need to register your phone number with for thedockerized signal messenger. Simply follow the instructions
in the terminal.You should see your produced messages on the kafka uihttp://localhost:8081/(useKAFKA__UI__PORTfrom.envfile).Implementation Details - How does it work?signal-kafka-producercallssrc/signalation/services/producer.pywhich has the following logic:It pools the Signal server and retrieves new Signal messages with their metadataIt produces the messages to a Kafka topic.If a message has an attachment, it downloads it and stores the file locally.
Additionally, corresponding metadata of the attachment is produced to a separate Kafka topic.Here is a more detailed description given by ChatGPT:Therunfunction is the main entry point of the code. It retrieves configuration settings from an environment file
and initializes a Kafka producer object. Then, it repeatedly calls therun_message_loop_iterationfunction,
which retrieves Signal messages and their attachments, produces them to the relevant Kafka topics, and sleeps for
a specified duration before it starts the next iteration.Thereceive_messagesfunction sends aGETrequest to the Signal server to retrieve new messages for a
given registered number. It returns a list of Signal messages, each represented by aSignalMessageobject.
Thereceive_attachmentsfunction sends a GET request to the Signal server to retrieve attachments associated
with a given list of messages. It returns a list ofAttachmentFileobjects, each representing a Signal attachment file.Theproduce_messagesfunction takes a list ofSignalMessageobjects and a Kafka producer object,
and produces them to the message topic.Finally, theEnhancedEncoderclass is a custom JSON encoder that converts Python objects into JSON strings.