Mastodon Streaming to Apache Pulsar via Python
Tags: apachepulsar, mastodon, python3
Published: 2023-01-18 00:42:43

---
title: Mastodon Streaming to Apache Pulsar via Python
published: true
date: 2023-01-18 00:40:00 UTC
tags: apachepulsar,mastodon,python3
canonical_url: 
---

[https://medium.com/@tspann/mastodon-streaming-to-pulsar-via-python-be7538112023](https://medium.com/@tspann/mastodon-streaming-to-pulsar-via-python-be7538112023)
  

  

<section style="box-sizing: inherit;"><div style="box-sizing: inherit;"><div style="box-sizing: inherit; overflow-wrap: break-word; word-break: break-word;">
<div style="box-sizing: inherit;"><h1 data-selectable-paragraph="" id="0dd8" style='box-sizing: inherit; color: #292929; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 32px; letter-spacing: -0.016em; line-height: 40px; margin: 0.6em 0px -0.27em;'>Mastodon Streaming to Pulsar via Python</h1></div>
<p data-selectable-paragraph="" id="47f9" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Python, Pulsar, Mastodon, Streaming, Real-Time, Events, Filtering</p>
<p data-selectable-paragraph="" id="2fc7" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'><span style="box-sizing: inherit; font-weight: 700;">Source</span>: <a href="https://github.com/tspannhw/pulsar-mastodon-sink" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit;" target="_blank">https://github.com/tspannhw/pulsar-mastodon-sink</a></p>
<p style="box-sizing: inherit; clear: both; margin: 56px auto 0px;"></p>
<div role="button" style="box-sizing: inherit; cursor: zoom-in; position: relative; transition: transform 300ms cubic-bezier(0.2, 0, 0.2, 1) 0s; width: 680px; z-index: auto;" tabindex="0"><div style="box-sizing: inherit; margin-left: auto; margin-right: auto; max-width: 3430px;"><img alt="" height="158" loading="eager" role="presentation" src="https://miro.medium.com/max/1400/1*XBO0CDIye5G6qG0gHhCCIQ.png" style="box-sizing: inherit; height: auto; max-width: 100%; vertical-align: middle; width: 680px;" width="700"></div></div>
<figcaption data-selectable-paragraph="" style='box-sizing: inherit; color: #757575; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; line-height: 20px; margin-left: auto; margin-right: auto; margin-top: 10px; max-width: 728px; text-align: center;'>Mastodon listener in Python streams to Pulsar</figcaption>
<p data-selectable-paragraph="" id="f634" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Ideally, the next step is either Apache Flink SQL or Apache Pinot and then some analytics with say Apache SuperSet. I am debating what to do with this data. I may add a full data pipeline with multiple Pulsar functions or could run some ML in Flink. I could stream it to a Pulsar sink that stores it to DeltaLake or Apache Iceberg. It would be interesting to run some NLP or ML on it or even feed it to <span style="box-sizing: inherit; font-weight: 700;">ChatGPT.</span></p>
<p style="box-sizing: inherit; clear: both; margin: 56px auto 0px;"></p>
<div role="button" style="box-sizing: inherit; cursor: zoom-in; position: relative; transition: transform 300ms cubic-bezier(0.2, 0, 0.2, 1) 0s; width: 680px; z-index: auto;" tabindex="0"><div style="box-sizing: inherit; margin-left: auto; margin-right: auto; max-width: 2161px;"><img alt="" height="560" loading="lazy" role="presentation" src="https://miro.medium.com/max/1400/1*a6OOz-qaxsGA4ozaXOMJdg.jpeg" style="box-sizing: inherit; height: auto; max-width: 100%; vertical-align: middle; width: 680px;" width="700"></div></div>
<p data-selectable-paragraph="" id="2867" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>I asked ChatGPT what I should do.</p>
<blockquote style="box-shadow: rgb(41, 41, 41) 3px 0px 0px 0px inset; box-sizing: inherit; margin: 0px 0px 0px -20px; padding-left: 23px;"><p data-selectable-paragraph="" id="24b6" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; font-style: italic; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>How should I process Mastodon social feeds with Apache Pulsar?</p></blockquote>
<p style="box-sizing: inherit; clear: both; margin: 56px auto 0px;"></p>
<div role="button" style="box-sizing: inherit; cursor: zoom-in; position: relative; transition: transform 300ms cubic-bezier(0.2, 0, 0.2, 1) 0s; width: 680px; z-index: auto;" tabindex="0"><div style="box-sizing: inherit; margin-left: auto; margin-right: auto; max-width: 1557px;"><img alt="" height="417" loading="lazy" role="presentation" src="https://miro.medium.com/max/1400/1*Ktx1Ge-DFOTuPxnt8MqVBw.jpeg" style="box-sizing: inherit; height: auto; max-width: 100%; vertical-align: middle; width: 680px;" width="700"></div></div>
<blockquote style="box-shadow: rgb(41, 41, 41) 3px 0px 0px 0px inset; box-sizing: inherit; margin: 0px 0px 0px -20px; padding-left: 23px;"><p data-selectable-paragraph="" id="173e" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; font-style: italic; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Can I stream Mastodon streaming data to Pulsar and then to Flink SQL for real-time analytics? If so, can you build me a SQL application that does that and joins it with another stream that is relevant.</p></blockquote>
<p style="box-sizing: inherit; clear: both; margin: 56px auto 0px;"></p>
<div role="button" style="box-sizing: inherit; cursor: zoom-in; position: relative; transition: transform 300ms cubic-bezier(0.2, 0, 0.2, 1) 0s; width: 680px; z-index: auto;" tabindex="0"><div style="box-sizing: inherit; margin-left: auto; margin-right: auto; max-width: 1492px;"><img alt="" height="627" loading="lazy" role="presentation" src="https://miro.medium.com/max/1400/1*sh-Y-BGrjcAhxMq3bjb1uw.jpeg" style="box-sizing: inherit; height: auto; max-width: 100%; vertical-align: middle; width: 680px;" width="700"></div></div>
<blockquote style="box-shadow: rgb(41, 41, 41) 3px 0px 0px 0px inset; box-sizing: inherit; margin: 0px 0px 0px -20px; padding-left: 23px;"><p data-selectable-paragraph="" id="91ef" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 21px; font-style: italic; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>write an application that streams mastodon streaming events to Apache Pulsar and aggregates # of toots with Flink SQL</p></blockquote>
<p data-selectable-paragraph="" id="7dcb" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>I’m sorry, I am not able to write a full-fledged application as it’s beyond the scope of my abilities and it would require a lot of resources and knowledge of your specific use-case and environment. However, I can provide you with a rough skeleton of how such an application could be built.</p>
<p data-selectable-paragraph="" id="5a00" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Here is a skeleton of a Flink job that streams Mastodon data from Pulsar and aggregates the number of toots using Flink SQL:</p>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="758a" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> org.apache.flink.streaming.api.scala._<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> org.apache.flink.table.api.{EnvironmentSettings, Table}<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> org.apache.flink.table.api.scala._<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">object MastodonTootCount {<br style="box-sizing: inherit;"> def <span style="box-sizing: inherit;">main</span><span style="box-sizing: inherit; color: #5c2699;">(args: Array[String])</span>: Unit = {<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #5c2699;">val</span> <span style="box-sizing: inherit; color: #3f6e74;">env</span> <span style="box-sizing: inherit;">=</span> StreamExecutionEnvironment.getExecutionEnvironment<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #5c2699;">val</span> <span style="box-sizing: inherit; color: #3f6e74;">settings</span> <span style="box-sizing: inherit;">=</span> EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build()<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #5c2699;">val</span> <span style="box-sizing: inherit; color: #3f6e74;">tEnv</span> <span style="box-sizing: inherit;">=</span> StreamTableEnvironment.create(env, settings)<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">// Connect to the Pulsar source, consume data from the topic and convert it to a DataStream</span><br style="box-sizing: inherit;"> val mastodonStream: DataStream[MastodonEvent] = env.addSource(<span style="box-sizing: inherit; color: #aa0d91;">new</span> <span style="box-sizing: inherit;">PulsarSource</span>[MastodonEvent](...))<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">// Convert the DataStream to a Table</span><br style="box-sizing: inherit;"> val mastodonTable: Table = tEnv.fromDataStream(mastodonStream)<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">// Register the table in the TableEnvironment</span><br style="box-sizing: inherit;"> tEnv.registerTable(<span style="box-sizing: inherit; color: #c41a16;">"mastodon"</span>, mastodonTable)<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">// Define the SQL query to aggregate the number of toots</span><br style="box-sizing: inherit;"> val tootCount: Table = tEnv.sqlQuery(<span style="box-sizing: inherit; color: #c41a16;">"SELECT COUNT(*) FROM mastodon WHERE event_type = 'toot'"</span>)<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">// Convert the result to a DataStream and print it</span><br style="box-sizing: inherit;"> val tootCountStream: DataStream[Long] = tEnv.toRetractStream[Long](tootCount).map(_._2)<br style="box-sizing: inherit;"> tootCountStream.print()<br style="box-sizing: inherit;"> env.execute(<span style="box-sizing: inherit; color: #c41a16;">"Mastodon Toot Count"</span>)<br style="box-sizing: inherit;"> }<br style="box-sizing: inherit;">}</span></pre>
<p data-selectable-paragraph="" id="2cfc" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>This is just a skeleton, you would need to fill in the details such as Pulsar configuration, data mapping and the specific logic for your use case. It’s important to note that this example uses the Flink Table and SQL API and assumes that the data received from Mastodon is of type MastodonEvent and has an event_type field, also it only considers toots and doesn’t cover other possible events. It would be beneficial to have a more detailed understanding of Mastodon’s streaming API, Pulsar’s features, and Flink’s SQL capabilities before proceeding.</p>
<p data-selectable-paragraph="" id="139b" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>For now, I am happy to get and stream the data as it streams through Mastondon.social. As a test I am publishing content that meets my simple filter. It will be posted here <a href="https://mastodon.social/@PaaSDev" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit;" target="_blank">https://mastodon.social/@PaaSDev</a></p>
<p data-selectable-paragraph="" id="d125" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>I saw an example of using NodeJS to stream Mastodon from Aiven and thought that looked fun. <a href="https://dev.to/ftisiot/stream-mastodon-data-to-apache-kafkar-using-nodejs-and-typescript-42mi-temp-slug-9035221" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit;" target="_blank">https://aiven.io/blog/mastodon-kafka-js</a> I wanted to do this in Python as that is a good language for Data Processing, Machine Learning, Pulsar, Kafka, Spark and other things. Perhaps next time I will use JavaScript, Go, C# or Java. If you have a suggestion drop it in a comment.</p>
<p data-selectable-paragraph="" id="e1c4" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Let’s look at the Python JSon Schema I setup for Pulsar.</p>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="00c1" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;"><span style="box-sizing: inherit; color: #aa0d91;">class</span> <span style="box-sizing: inherit;">mastodondata</span>(<span style="box-sizing: inherit;">Record</span>):<br style="box-sizing: inherit;"> language = String()<br style="box-sizing: inherit;"> created_at = String()<br style="box-sizing: inherit;"> ts = Float()<br style="box-sizing: inherit;"> uuid = String()<br style="box-sizing: inherit;"> uri = String()<br style="box-sizing: inherit;"> url = String()<br style="box-sizing: inherit;"> favourites_count = Integer()<br style="box-sizing: inherit;"> replies_count = Integer()<br style="box-sizing: inherit;"> reblogs_count = Integer()<br style="box-sizing: inherit;"> content = String()<br style="box-sizing: inherit;"> username = String()<br style="box-sizing: inherit;"> accountname = String()<br style="box-sizing: inherit;"> displayname = String()<br style="box-sizing: inherit;"> note = String()<br style="box-sizing: inherit;"> followers_count = Integer()<br style="box-sizing: inherit;"> statuses_count = Integer()</span></pre>
<p data-selectable-paragraph="" id="afc5" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>An example result consumed with the command-line client in Apache Pulsar. I should use Pulsar Shell next time.</p>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="a6b6" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;">----- got message -----<br style="box-sizing: inherit;">key<span style="box-sizing: inherit;">:</span><span style="box-sizing: inherit;">[</span><span style="box-sizing: inherit; color: #1c00cf;">20230117223725</span>_f68a866e<span style="box-sizing: inherit; color: #1c00cf;">-1</span>df1<span style="box-sizing: inherit; color: #1c00cf;">-4</span>d5d-b977-eb151535f240<span style="box-sizing: inherit;">]</span><span style="box-sizing: inherit;">,</span> properties<span style="box-sizing: inherit;">:</span><span style="box-sizing: inherit;">[</span><span style="box-sizing: inherit;">]</span><span style="box-sizing: inherit;">,</span> content<span style="box-sizing: inherit;">:</span><span style="box-sizing: inherit;">{</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"language"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"en"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"created_at"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"2023-01-17 22:37:25.347000+00:00"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"ts"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">20230117223725.0</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"uuid"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"20230117223725_f68a866e-1df1-4d5d-b977-eb151535f240"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"uri"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"https://mastodon.social/users/PaaSDev/statuses/109706939291797415"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"url"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"https://mastodon.social/@PaaSDev/109706939291797415"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"favourites_count"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">0</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"replies_count"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">0</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"reblogs_count"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">0</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"content"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"&lt;p&gt;I am working on an Apache Pulsar streaming application in Python to ingest mastodon messages. &lt;a href=\"https://github.com/tspannhw/pulsar-mastodon-sink\" target=\"_blank\" rel=\"nofollow noopener noreferrer\"&gt;&lt;span class=\"invisible\"&gt;https://&lt;/span&gt;&lt;span class=\"ellipsis\"&gt;github.com/tspannhw/pulsar-mas&lt;/span&gt;&lt;span class=\"invisible\"&gt;todon-sink&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"username"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"PaaSDev"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"accountname"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">"PaaSDev"</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"displayname"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">""</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"note"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #c41a16;">""</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"followers_count"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">0</span><span style="box-sizing: inherit;">,</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #836c28;">"statuses_count"</span><span style="box-sizing: inherit;">:</span> <span style="box-sizing: inherit; color: #1c00cf;">1</span><br style="box-sizing: inherit;"><span style="box-sizing: inherit;">}</span></span></pre>
<p data-selectable-paragraph="" id="4c36" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>Let’s run the final application.</p>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="450b" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;">python3 stream.py<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">2023-01-17 17:28:17.197 INFO [0x16ec2b000] HandlerBase:72 | [persistent://public/default/mastodon-partition-0,] Getting connection from pool<br style="box-sizing: inherit;">2023-01-17 17:28:17.200 INFO [0x16ec2b000] ProducerImpl:190 | [persistent://public/default/mastodon-partition-0,] Created producer on broker [127.0.0.1:56776 -&gt; 127.0.0.1:6650]<br style="box-sizing: inherit;">20230117222817.0</span></pre>
<p data-selectable-paragraph="" id="4acf" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>The Python code is below.</p>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="c8d2" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> mastodon<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> mastodon <span style="box-sizing: inherit; color: #aa0d91;">import</span> Mastodon<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> pprint <span style="box-sizing: inherit; color: #aa0d91;">import</span> pprint<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> requests<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> bs4 <span style="box-sizing: inherit; color: #aa0d91;">import</span> BeautifulSoup<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> pulsar<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> pulsar.schema <span style="box-sizing: inherit; color: #aa0d91;">import</span> *<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> time<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> sys<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> datetime<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> subprocess<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> sys<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> os<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> subprocess <span style="box-sizing: inherit; color: #aa0d91;">import</span> PIPE, Popen<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> traceback<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> math<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> base64<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> json<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> time <span style="box-sizing: inherit; color: #aa0d91;">import</span> gmtime, strftime<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> random, string<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> psutil<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> uuid<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> json<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> socket <br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> logging<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">from</span> jsonpath_ng <span style="box-sizing: inherit; color: #aa0d91;">import</span> jsonpath, parse<br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">import</span> re<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">#### Apache Pulsar</span><br style="box-sizing: inherit;">pulsarClient = pulsar.Client(<span style="box-sizing: inherit; color: #c41a16;">'pulsar://localhost:6650'</span>)<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">#### Schema Record</span><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">class</span> <span style="box-sizing: inherit;">mastodondata</span>(<span style="box-sizing: inherit;">Record</span>):<br style="box-sizing: inherit;"> language = String()<br style="box-sizing: inherit;"> created_at = String()<br style="box-sizing: inherit;"> ts = Float()<br style="box-sizing: inherit;"> uuid = String()<br style="box-sizing: inherit;"> uri = String()<br style="box-sizing: inherit;"> url = String()<br style="box-sizing: inherit;"> favourites_count = Integer()<br style="box-sizing: inherit;"> replies_count = Integer()<br style="box-sizing: inherit;"> reblogs_count = Integer()<br style="box-sizing: inherit;"> content = String()<br style="box-sizing: inherit;"> username = String()<br style="box-sizing: inherit;"> accountname = String()<br style="box-sizing: inherit;"> displayname = String()<br style="box-sizing: inherit;"> note = String()<br style="box-sizing: inherit;"> followers_count = Integer()<br style="box-sizing: inherit;"> statuses_count = Integer()<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">#### Keywords to match</span><br style="box-sizing: inherit;">keywordList = [<span style="box-sizing: inherit; color: #c41a16;">'apache spark'</span>,<span style="box-sizing: inherit; color: #c41a16;">'Apache Spark'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Apache Pinot'</span>,<span style="box-sizing: inherit; color: #c41a16;">'flink'</span>,<span style="box-sizing: inherit; color: #c41a16;">'Flink'</span>,<span style="box-sizing: inherit; color: #c41a16;">'Apache Flink'</span>,<span style="box-sizing: inherit; color: #c41a16;">'kafka'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Kafka'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Apache Kafka'</span>, <span style="box-sizing: inherit; color: #c41a16;">'pulsar'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Pulsar'</span>, <span style="box-sizing: inherit; color: #c41a16;">'datapipeline'</span>, <span style="box-sizing: inherit; color: #c41a16;">'real-time'</span>, <span style="box-sizing: inherit; color: #c41a16;">'real-time streaming'</span>, <span style="box-sizing: inherit; color: #c41a16;">'StreamNative'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Confluent'</span>, <span style="box-sizing: inherit; color: #c41a16;">'RedPandaData'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Apache Pulsar'</span>, <span style="box-sizing: inherit; color: #c41a16;">'streaming'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Streaming'</span>, <span style="box-sizing: inherit; color: #c41a16;">'big data'</span>, <span style="box-sizing: inherit; color: #c41a16;">'Big Data'</span>]<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">#### Build our Regex</span><br style="box-sizing: inherit;">words_re = re.<span style="box-sizing: inherit; color: #5c2699;">compile</span>(<span style="box-sizing: inherit; color: #c41a16;">"|"</span>.join(keywordList))<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #007400;">#### Listener for Mastodon events</span><br style="box-sizing: inherit;"><br style="box-sizing: inherit;"><span style="box-sizing: inherit; color: #aa0d91;">class</span> <span style="box-sizing: inherit;">Listener</span>(mastodon.StreamListener):<br style="box-sizing: inherit;"><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #aa0d91;">def</span> <span style="box-sizing: inherit;">on_update</span>(<span style="box-sizing: inherit; color: #5c2699;">self, status</span>):<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #aa0d91;">if</span> words_re.search(status.content):<br style="box-sizing: inherit;"> pulsarProducer = pulsarClient.create_producer(topic=<span style="box-sizing: inherit; color: #c41a16;">'persistent://public/default/mastodon'</span>,<br style="box-sizing: inherit;"> schema=JsonSchema(mastodondata), properties={<span style="box-sizing: inherit; color: #c41a16;">"producer-name"</span>: <span style="box-sizing: inherit; color: #c41a16;">"mastodon-py-strean"</span>,<span style="box-sizing: inherit; color: #c41a16;">"producer-id"</span>: <span style="box-sizing: inherit; color: #c41a16;">"mastodon-producer"</span> })<br style="box-sizing: inherit;"> mastodonRec = mastodondata()<br style="box-sizing: inherit;"> uuid_key = <span style="box-sizing: inherit; color: #c41a16;">'{0}_{1}'</span>.<span style="box-sizing: inherit; color: #5c2699;">format</span>(strftime(<span style="box-sizing: inherit; color: #c41a16;">"%Y%m%d%H%M%S"</span>,gmtime()),uuid.uuid4())<br style="box-sizing: inherit;"> mastodonRec.language = status.language<br style="box-sizing: inherit;"> mastodonRec.created_at = <span style="box-sizing: inherit; color: #5c2699;">str</span>(status.created_at)<br style="box-sizing: inherit;"> mastodonRec.ts = <span style="box-sizing: inherit; color: #5c2699;">float</span>(strftime(<span style="box-sizing: inherit; color: #c41a16;">"%Y%m%d%H%M%S"</span>,gmtime()))<br style="box-sizing: inherit;"> mastodonRec.uuid = uuid_key<br style="box-sizing: inherit;"> mastodonRec.uri = status.uri<br style="box-sizing: inherit;"> mastodonRec.url = status.url<br style="box-sizing: inherit;"> mastodonRec.favourites_count = status.favourites_count<br style="box-sizing: inherit;"> mastodonRec.replies_count = status.replies_count<br style="box-sizing: inherit;"> mastodonRec.reblogs_count = status.reblogs_count<br style="box-sizing: inherit;"> mastodonRec.content = status.content <br style="box-sizing: inherit;"> mastodonRec.username = status.account.username<br style="box-sizing: inherit;"> mastodonRec.accountname = status.account.acct<br style="box-sizing: inherit;"> mastodonRec.displayname = status.account.display_name<br style="box-sizing: inherit;"> mastodonRec.note = status.account.note<br style="box-sizing: inherit;"> mastodonRec.followers_count = status.account.followers_count<br style="box-sizing: inherit;"> mastodonRec.statuses_count = status.account.statuses_count<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #5c2699;">print</span>(mastodonRec.ts)<br style="box-sizing: inherit;"> pulsarProducer.send(mastodonRec,partition_key=<span style="box-sizing: inherit; color: #5c2699;">str</span>(uuid_key))<br style="box-sizing: inherit;"> pulsarProducer.flush()<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">#producer.send('rp4-kafka-1', mastodonRec.encode('utf-8'))</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;">#producer.flush()</span><br style="box-sizing: inherit;"><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #aa0d91;">def</span> <span style="box-sizing: inherit;">on_notification</span>(<span style="box-sizing: inherit; color: #5c2699;">self, notification</span>):<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #007400;"># print(f"on_notification: {notification}")</span><br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #5c2699;">print</span>(<span style="box-sizing: inherit; color: #c41a16;">"notification"</span>)<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">Mastodon.create_app(<br style="box-sizing: inherit;"> <span style="box-sizing: inherit; color: #c41a16;">'streamreader'</span>,<br style="box-sizing: inherit;"> api_base_url = <span style="box-sizing: inherit; color: #c41a16;">'https://mastodon.social'</span><br style="box-sizing: inherit;">)<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">mastodon = Mastodon(api_base_url=<span style="box-sizing: inherit; color: #c41a16;">'https://mastodon.social'</span>)<br style="box-sizing: inherit;">mastodon.stream_public(Listener())</span></pre>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://mastodon.social/@PaaSDev" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Timothy Spann (@PaaSDev@mastodon.social)</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>6 Posts, 1 Following, 0 Followers · Developer Advocate for Streaming</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>mastodon.social</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*lmMsnKV6ejpTYTVw"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<p data-selectable-paragraph="" id="90af" style='box-sizing: inherit; color: #292929; font-family: source-serif-pro, Georgia, Cambria, "Times New Roman", Times, serif; font-size: 20px; letter-spacing: -0.003em; line-height: 32px; margin: 2em 0px -0.46em; word-break: break-word;'>In the follow-up I will decide what to do with this data, where it should go and if it needs additional data fields. I think I may add some TCP/IP status data and perhaps join in some other streams in Flink. I think depending on the result of our NLP and Sentiment analytics we could add other data lookups dynamically.</p>
<h1 data-selectable-paragraph="" id="5ef3" style='box-sizing: inherit; color: #292929; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 22px; letter-spacing: 0px; line-height: 28px; margin: 3.14em 0px -0.37em;'>Pulsar Commands</h1>
<pre style='background: rgb(249, 249, 249); border-radius: 4px; border: 1px solid rgb(107, 107, 107); box-sizing: inherit; color: #292929; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; margin-bottom: 0px; margin-top: 56px; overflow-x: auto; padding: 32px;'><span data-selectable-paragraph="" id="8397" style="box-sizing: inherit; display: block; font-size: 14px; letter-spacing: -0.022em; line-height: 1.4; margin-bottom: -0.2em; margin-top: -0.2em; min-width: fit-content;"><br style="box-sizing: inherit;">pulsar<span style="box-sizing: inherit;">-</span>admin topics <span style="box-sizing: inherit; color: #aa0d91;">create</span> persistent:<span style="box-sizing: inherit;">/</span><span style="box-sizing: inherit;">/</span>public<span style="box-sizing: inherit;">/</span><span style="box-sizing: inherit; color: #aa0d91;">default</span><span style="box-sizing: inherit;">/</span>mastodon<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">pulsar<span style="box-sizing: inherit;">-</span>admin schemas <span style="box-sizing: inherit; color: #aa0d91;">get</span> persistent:<span style="box-sizing: inherit;">/</span><span style="box-sizing: inherit;">/</span>public<span style="box-sizing: inherit;">/</span><span style="box-sizing: inherit; color: #aa0d91;">default</span><span style="box-sizing: inherit;">/</span>mastodon<br style="box-sizing: inherit;"><br style="box-sizing: inherit;">pulsar<span style="box-sizing: inherit;">-</span>client consume "persistent://public/default/mastodon" <span style="box-sizing: inherit;">-</span>s "mreader2" <span style="box-sizing: inherit;">-</span>n <span style="box-sizing: inherit; color: #1c00cf;">0</span><br style="box-sizing: inherit;"></span></pre>
<h1 data-selectable-paragraph="" id="1eb5" style='box-sizing: inherit; color: #292929; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 22px; letter-spacing: 0px; line-height: 28px; margin: 3.14em 0px -0.37em;'><span style="box-sizing: inherit; font-weight: inherit;">References</span></h1>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/FLiP-Pi-Iceberg-Thermal" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - tspannhw/FLiP-Pi-Iceberg-Thermal: Apache Iceberg + Apache Pulsar + Thermal Sensor Data…</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Apache Iceberg + Apache Pulsar + Thermal Sensor Data from a Raspberry Pi Run Apache Pulsar 2.10.2 (standalone, docker…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*St4DVVqt20lHMo24"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/pulsar-mastodon-sink" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - tspannhw/pulsar-mastodon-sink: Mastodon data streaming</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Mastodon data streaming. Contribute to tspannhw/pulsar-mastodon-sink development by creating an account on GitHub.</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*BFpYcOu44nHEpIvA"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/halcy/Mastodon.py" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - halcy/Mastodon.py: Python wrapper for the Mastodon ( https://github.com/mastodon/mastodon/…</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Python wrapper for the Mastodon ( https://github.com/mastodon/mastodon/ ) API. Feature complete for public API as of…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*fd-BecVDauWDNeMP"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://martinheinz.dev/blog/86" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;"><div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Getting Started with Mastodon API in Python</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>With what's happening in Twitter, many people are considering moving to Mastodon . Like Twitter, Mastodon also has an…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>martinheinz.dev</p></div>
</div></div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/FLiPN-Py-Stocks" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - tspannhw/FLiPN-Py-Stocks: finnhub stocks</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>finnhub stocks. Contribute to tspannhw/FLiPN-Py-Stocks development by creating an account on GitHub.</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*yZc5Rflie0kQUFgA"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/pulsar-thermal-pinot" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - tspannhw/pulsar-thermal-pinot: Apache Pulsar - Apache Pinot - Thermal Sensor Data</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Apache Pulsar - Apache Pinot - Thermal Sensor Data…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*B8pzewU1ADUN-HkP"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/FLiP-Current22-LetsMonitorAllTheThings" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>GitHub - tspannhw/FLiP-Current22-LetsMonitorAllTheThings: Let's Monitor The Conditions at the…</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>Session Time: 11:15 am - 12:00 pm Session Date: Wednesday, 5 October 2022 Session Type: In-Person Location: Ballroom G…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*MOk62fijwwtMLItk"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
<div style="box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; margin-top: 32px;"><a href="https://github.com/tspannhw/pulsar-transit-function/blob/main/src/main/java/dev/pulsarfunction/transit/NLPService.java" rel="noopener ugc nofollow" style="-webkit-tap-highlight-color: transparent; box-sizing: inherit; text-decoration-line: none;" target="_blank"><div style="box-sizing: inherit; display: flex; flex: 0 0 auto; padding: 0px;">
<div style="box-sizing: inherit; display: flex; flex-direction: column; flex: 1 1 auto; justify-content: center; padding: 16px 20px;">
<h2 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #292929; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; letter-spacing: 0px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>pulsar-transit-function/NLPService.java at main · tspannhw/pulsar-transit-function</h2>
<div style="box-sizing: inherit; margin-top: 8px;"><h3 style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px; font-weight: 400; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>You can't perform that action at this time. You signed in with another tab or window. You signed out in another tab or…</h3></div>
<div style="box-sizing: inherit; margin-top: 12px;"><p style='-webkit-box-orient: vertical; -webkit-line-clamp: 2; box-sizing: inherit; color: #757575; display: -webkit-box; font-family: sohne, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 13px; line-height: 20px; margin: 0px; max-height: 40px; overflow: hidden; text-overflow: ellipsis;'>github.com</p></div>
</div>
<div style="box-sizing: inherit; width: 160px;"><div style='background-image: url("https://miro.medium.com/max/320/0*M7WVSJkHeCn-8u0K"); background-origin: border-box; background-position: 50% 50%; background-size: cover; box-shadow: rgb(230, 230, 230) 0px 0px 0px 1px inset; box-sizing: inherit; height: 167px; max-width: 100%; width: 160px;'></div></div>
</div></a></div>
</div></div></section>

[
Apache Pulsar
](https://medium.com/tag/apache-pulsar?source=post_page-----be7538112023---------------apache_pulsar-----------------)

[
Chatgpt
](https://medium.com/tag/chatgpt?source=post_page-----be7538112023---------------chatgpt-----------------)

[
Open Source
](https://medium.com/tag/open-source?source=post_page-----be7538112023---------------open_source-----------------)

[
Python
](https://medium.com/tag/python?source=post_page-----be7538112023---------------python-----------------)

[
Mastodon
](https://medium.com/tag/mastodon?source=post_page-----be7538112023---------------mastodon-----------------)