Package: geniusrise-listeners
ListenersThis is a collection of generic streaming listeners (Spouts).Table of ContentsListenersUsageWebhooksKafkaWebsocketsUDPQUICREST API pollingRabbitMQ / AMQPMQTTRedis Pub-SubRedis StreamsAWS SNSAWS SQSsocket.ioActiveMQKinesisGrpcZeroMQIncludes:No.NameDescriptionOutput TypeInput Type1WebhookCherrypy server that accepts all HTTP API callsStreamingHTTP2KafkaKafka client that listens to a topicStreamingKafka3WebsocketWebsocket server that listens to a socketStreamingWebsocket4UDPUDP server that listens to a given portStreamingUDP5QUICAioquic server that listens to a given portStreamingQUIC6HTTP PollingHTTP client that keeps polling an APIStreamingHTTP7RabbitMQ / AMQPRabbitMQ client that listens to a given queueStreamingAMQP8MQTTMQTT client that subscribes and listens to a topicStreamingMQTT9Redis Pub-SubRedis client that subscribes to a Pub/Sub channelStreamingRedis Pub-Sub10Redis StreamsRedis client that listens to a streamStreamingRedis Streams11AWS SNSAWS client that listens to SNS notificationsStreamingAWS SNS12AWS SQSAWS client that listens to messages from an SQS queueStreamingAWS SQS13SocketIoSocketIo client that listens to a namespaceStreamingSocketIo14ActiveMQActiveMQ client that listens to a queueStreamingActiveMQ15KinesisKinesis client that listens to a streamStreamingKinesis16GrpcgRPC client that listens to a serverStreaminggRPC17ZeroMQZeroMQ client that listens to a topicStreamingZeroMQUsageTo test, first bring up all related services via the supplied docker-compose:dockercomposeup-d
dockercomposelogs-fThese management consoles will be available:ConsoleLinkKafka UIhttp://localhost:8088/RabbitMQ UIhttp://localhost:15672/Localstack APIhttp://localhost:4566Postgres can be accessed with:dockerexec-itgeniusrise-postgres-1psql-UpostgresWebhooksgeniusWebhookrise\streaming\--output_kafka_topicwebhook_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsport=3001Test:curl-XPOST\-H"Content-Type: application/json"\-d'{"lol": "teeeestss"}'\http://localhost:3001KafkageniusKafkarise\streaming\--output_kafka_topickafka_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argstopic=kafka_test_inputgroup_id=kafka_testbootstrap_servers=localhost:9094WebsocketsgeniusWebsocketrise\streaming\--output_kafka_topicwebsocket_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argshost=localhostport=3002Test:cargoinstallwebsocatecho'{"lol": "heheheheheheheh"}'|websocatws://localhost:3002UDPgeniusUdprise\streaming\--output_kafka_topicudp_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argshost=localhostport=3003Test:yay-Snetcatecho-n'{"key": "value"}'|nc-u-w1localhost3003QUICopensslgenpkey-algorithmRSA-out~/.ssh/quic_key.pem
opensslreq-new-x509-key~/.ssh/quic_key.pem-out~/.ssh/quic_cert.pem-days365-subj"/CN=localhost"geniusQuicrise\streaming\--output_kafka_topicudp_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--args\cert_path=/home/ixaxaar/.ssh/quic_cert.pem\key_path=/home/ixaxaar/.ssh/quic_key.pem\host=localhost\port=3004Test:go install github.com/spacewander/quick@latest

quick -insecure https://localhost:3004REST API pollinggeniusRESTAPIPollrise\streaming\--output_kafka_topicpoll_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--args\url=https://reqres.in/api/users\method=GET\interval=6\body=""\headers='{"content-type": "application/json"}'\params='{"page": 2}'RabbitMQ / AMQPgeniusRabbitMQrise\streaming\--output_kafka_topicrabbitmq_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsqueue_name=geniusrise_testhost=localhostusername=adminpassword=adminTest:Gotohttp://localhost:15672/#/queues/%2F/geniusrise_testMQTTgeniusMQTTrise\streaming\--output_kafka_topicmqtt_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argshost=localhostport=1883topic=geniusrise_testsnapinstallmqtt-explorer# GUI, create a topic, send a messageordockerexec-itstreaming-spouts-mosquitto-1mosquitto_pub-h127.0.0.1-t"geniusrise_test"-m'{"test": "mqtt message"}'Redis Pub-SubgeniusRedisPubSubrise\streaming\--output_kafka_topicredispubsub_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argschannel=geniusrise_testhost=localhostport=6380Test:redis-cliPUBLISHgeniusrise_test'{"test": "redis pubsub message"}'Redis StreamsgeniusRedisStreamrise\streaming\--output_kafka_topicredisstream_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsstream_key=geniusrise_testhost=localhostTest:redis-cliXADDgeniusrise_test*test"redis stream message"AWS SNSgeniusSNSrise\streaming\--output_kafka_topicsns_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listenTest:awssnscreate-topic--namegeniusrise_test
awssnspublish--topic-arnarn:aws:sns:ap-south-1:866011655254:geniusrise_test--message'{"test": "sns message"}'AWS SQSgeniusSQSrise\streaming\--output_kafka_topicsqs_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsqueue_url=https://sqs.ap-south-1.amazonaws.com/866011655254/geniusrise_testTest:awssqssend-message--queue-urlhttps://sqs.ap-south-1.amazonaws.com/866011655254/geniusrise_test--message-body'{"test": "sqs message"}'socket.iogeniusSocketIorise\streaming\--output_kafka_topicsocketio_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsurl=http://localhost:3000namespace=/chatTest:# Use a SocketIo client to emit a message to the specified namespace.ActiveMQgeniusActiveMQrise\streaming\--output_kafka_topicactivemq_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argshost=localhostport=61613destination=my_queueTest:# Use an ActiveMQ client to send a message to the specified destination.KinesisgeniusKinesisrise\streaming\--output_kafka_topickinesis_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsstream_name=my_streamshard_id=shardId-000000000000Test:# Use the AWS CLI or SDK to put a record into the specified Kinesis stream.GrpcgeniusGrpcrise\streaming\--output_kafka_topicgrpc_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsserver_address=localhost:50051request_data=my_requestsyntax=proto3Test:# Use a gRPC client to send a message to the specified server address.ZeroMQgeniusZeroMQrise\streaming\--output_kafka_topiczmq_test\--output_kafka_cluster_connection_stringlocalhost:9094\postgres\--postgres_host127.0.0.1\--postgres_port5432\--postgres_userpostgres\--postgres_passwordpostgres\--postgres_databasegeniusrise\--postgres_tablestate\listen\--argsendpoint=tcp://localhost:5555topic=my_topicsyntax=jsonTest:# Use a ZeroMQ client to send a message to the specified endpoint and topic.